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ABSTRACT 


This  dissertation  considers  the  problem  (usually  called  the  shaping 
filter  problem)  of  synthesizing  linear  systems  whose  responses  to  white  noise 
input  processes  and  appropriate  sets  of  random  initial  conditions  will  be 
stochastic  processes  whose  covariance  functions  are  prescribed  functions  of 
two  variables* 

A  detailed  review  of  previous  work  on  this  problem  is  presented  and  the 
limitations  of  and  the  errors  in  this  previous  work  are  carefully  pointed  out* 

The  properties  of  weighting  functions  of  systems  characteri zabl e  by  finite- 
order  linear  differential  equations  are  developed  in  detail  and  these  results 
are  used  to  develop  the  properties  of  the  covariance  functions  of  the  responses 
of  such  systems  to  white  noise  inputs  and  random  initial  conditions* 

Based  on  this  work,  exact  solutions  to  the  shaping  filter  problem  are 
presented  for  certain  special  cases  and  some  discussion  of  predictability  of  the 
processes  is  presented*  Attention  is  then  focused  on  the  general  problem  and  by 
means  of  the  Schauder  Fixed  Point  Theorem  and  by  Picard's  method  of  successive 
approximations  the  existence  of  physically  realizable  shaping  filters  is  estab¬ 
lished  for  a  large  class  of  separable  covariance  functions*  The  question  of  the 
uniqueness  of  the  shaping  filter  and  its  relationship  to  the  covariance  matrix 
of  the  set  of  random  initial  conditions  is  investigated*  It  is  shown  that  if 
an  appropriate  set  of  random  initial  conditions  is  specified,  then  the  weighting 
function  of  the  shaping  filter  is  unique  up  to  a  multiplicative  factor  of  ±  1* 

The  further  requirements  on  the  covariance  function  in  order  to  guarantee  that 
the  shaping  filter  can  be  characterized  by  a  finite-order,  linear  differential 
equation  with  continuous  coefficients  are  given  as  well  as  certain  leaser  re¬ 
quirements  which  permit  easy  analog  simulation* 

Some  brief  comments  are  made  relative  to  computational  requirements  and 
how  they  can  best  be  carried  out  and  the  pertinent  references  are  cited*  Finally 
the  two  main  areas  of  applications  of  shaping  filters  are  briefly  outlined. 


11/ 


TABLE  OF  CONTENTS 


N|t 

CHAPTER  1  -  INTRODUCTION 

1.1  A  General  Description  of  the  Problem . .  1 

1.2  Historical  Review  of  the  Shaping  Filter  Problem .  2 

1.2.1  Summary  of  Previous  Work  for  Stationary  Processes .  3 

1.2.2  Summary  of  the  Work  of  Darlington  . 6 

1.2.3  Summary  of  the  Work  of  Batkov . 11 

1.2.4  Summary  of  the  Work  of  Leonov . 16 

CHAPTER  2  -  PROPERTIES  OF  WEIGHTING  FUNCTIONS 
FOR  A  CLASS  OF  LINEAR  SYSTEMS 

2.1  Introduction . 21 

2.2  Properties  in  the  Region  Where  t  =  r .  22 

2.3  Properties  of  the  Discontinuities  of  JN(t,r)  and 

Its  Partial  Derivatives  at  t  =  r . . . 25 

2.4  A  Synthesis  Problem .  27 

CHAPTER  3  -  PROPERTIES  OF  THE  COVARIANCE  FUNCTIONS 
OF  A  CLASS  OF  STOCHASTIC  PROCESSES 

3.1  Introduction . 29 

3.2  Some  General  Properties .  29 

3.3  Discontinuity  Properties .  30 

3.4  A  Final  Identity .  32 

3.5  A  Discussion  of  Batkov*  s  Error . . .  33 

CHAPTER  4  -  EXACT  SOLUTION  OF  THE  SHAPING  FILTER  PROBLEM 
FOR  SOME  SPECIAL  CASES 

4.1  Introduction  .  37 

4.2  The  ”  Simplest  ”  Case .  37 

4.3  The1' Almost  Stationary”  Case .  43 

4.4  The  ”  Nondegenerate”  Case .  44 

CHAPTER  5  -  SOME  FUNDAMENTAL  RESULTS  ON  EXISTENCE  AND  UNIQUENESS 

5.1  Introduction .  47 

5.2  Casting  and  Recasting  the  Problem .  48 

5.3  Discussion  of  Fixed  Point  Theorems .  51 

5.4  Resolution  of  the  Existence  Problem .  52 

5.5  The  Question  of  Uniqueness  of  the  Shaping  Filter .  54 

CHAPTER  6  -  COMPUTATIONAL  ASPECTS  AND  APPLICATIONS 

6.1  Computational  Aspects .  59 

6*2  Applications . - .  60 

REFERENCES .  65 

APPENDIX  1 .  67 


v 


CHAPTER  1 


INTRODUCTION 

1.1  A  GENERAL  DESCRIPTION  OF  THE  PROBLEM 

A  problem  of  some  current  interest  in  the  field  of  stochastic  processes 
can  be  described  briefly  as  follows,  (see  Figure  1). 

RANDOM  INITIAL 
CONDITIONS 


FIGURE  1 


Given  the  covariance  function  r(tt,t2)  of  some  continuous  in  the  mean  stochas¬ 
tic  process^  determine  the  weighting  function  and/or  differential  equation,  if 
appropriate,  of  a  continuous,  physically  realizable,  linear  system  and  the  co- 
variances  of  a  set  of  random  initial  conditions,  if  required,  such  that  the 
response  {X(t)}  of  the  system  to  a  white  noise  input  process  {K(t)}  and  the  set 
of  random  initial  conditions  will  be  a  stochastic  process  whose  covariance 
function  satisfies  the  equality  =  ^  *  i  #  *  2  ^  *  This  problem 

which  will  be  referred  to  from  hereon  as  the  shaping  filter  problem,*  has  not 
yet  been  solved  for  the  general  case  where  r(t,,t2)  is  an  arbitrary  continuous 
covariance  function;  i,e,,  an  arbitrary,  bounded,  nonnegative-definite,  con¬ 
tinuous  function  defined  on  a  region  T  X  T  of  the  real  plane  where  T  is  an 
interval  of  the  real  line.  However,  by  suitably  restricting  the  class  of  ad¬ 
missible  covariance  functions  and  the  interval  T,  certain  fairly  definitive 
results  have  been  obtained.  These  results  are  summarized  in  the  following 
section. 

This  dissertation  is  devoted  to  a  study  of  the  shaping  filter  problem 
for  the  class  of  separable  covariance  functions,  where  T  is  a  finite  or  semi- 
infinite  (to  the  right)  interval,  and  to  the  relationship  between  the  properties 
of  the  covariance  function  and  the  possibility  of  characterizing  the  shapir.g 
filter  by  a  finite-order,  linear  differential  equation.  Particular  emphasis 
is  placed  on  the  goal  of  obtaining  the  solution  to  the  problem  in  a  form 

*There  is,  of  course,  a  related  problem  for  discrete  paraawter  stochastic  processes;  but,  since  these 
two  probiesis  are  quite  closely  related,  attention  herein  is  directed  primarily  to  the  continuous 
parameter  case. 
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directly  applicable  in  engineering  practice,  as  is  proper  for  a  dissertation 
in  engineering.  Finally,  the  applications  of  the  solution  to  the  shaping 
filter  problem  to  the  problems  of  analog  simulation  and  linear,  1  east- squares 
filtering  is  discussed. 

1,2  HISTORICAL  REVIEW  OF  TIIE  SHAPING  FILTER  PROBLEM 

In  order  to  place  the  results  obtained  in  this  dissertation  in  proper 
perspective,  it  is  necessary  to  have  a  precise  summary  of  all  significant  pre¬ 
viously  published  works  on  the  shaping  filter  problem.  However,  before  such 
a  summary  is  given,  a  brief  chronological  sketch  of  this  work  is  in  order. 

The  first  significant  result  relative  to  the  shaping  filter  problem  was 
apparently  obtained  by  Wold  1 1 ,  19  38]  for  discrete  parameter  stationary*  pro¬ 
cesses  and  is  part  of  his  fundamental  decomposition  theorem  [l,p.89]»  Kolmogorov 
[2,1939;  3,1941;  4,194l]  then  put  Wold's  decomposition  theorem  in  an  analytic 
setting  and  obtained  some  new  theorems  for  discrete  parameter  stationary  pro¬ 
cesses,  parts  of  which  again  pertain  to  the  shaping  filter  problem.  Indepen¬ 
dently,  Wiener  [5,1942]  obtained  Kolmogorov’s  results  for  discrete  parameter 
stationary  processes  with  absolutely  continuous  spectral  distribution  functions 
and  generalized  the  results  to  include  continuous  parameter  stationary  processes 
with  absolutely  continuous  spectral  distribution  functions.  Wiener  obtained 
thereby,  as  part  of  his  results,  the  first  solution  of  the  shaping  filter  pro¬ 
blem  for  continuous  parameter  stationary  processes.  Then  Hanner  [6,1949]  and 
Karhumen  [7,1950]  obtained,  by  different  methods,  the  continuous  parameter 
analog  of  the  Wold  decomposition  theorem,  parts  of  which  again  pertain  to  the 
shaping  filter  problem.  Bode  and  Shannon  [8,19 5 0] ,  in  their  simplified  heu¬ 
ristic  derivation  of  Wiener's  results  on  linear,  least-squares,  prediction  and 
filtering  theory  stressed  the  solution  of  the  shaping  filter  problem  as  an 
important  step  in  their  method.  The  next  significant  result  relative  to  the 
shaping  filter  problem  was  obtained  by  Darlington  [9,1959]  in  his  generaliza¬ 
tion  of  Bode  and  Shannon's  work  so  as  to  include  nonstationary  processes. 

While  Darlington  did  make  a  good  start  on  the  problem  for  continuous  parameter 
nonstationary  processes,  he  did  not  obtain  very  useful  results  as  will  be  clear 
from  later  discussion.  Two  months  later,  Batkov  [10,1959]  published  a  paper 
which  presents  three  methods  for  solving  the  shaping  filter  problem,  including 
an  algebraic  method  using  various  partial  derivatives  of  the  covariance  func¬ 
tion,  for  a  certain  class  of  continuous  parameter  nonstationary  processes.  As 
will  be  shown  later,  Baikov's  algebraic  procedure  only  works  for  a  rather 

* Stationary  should  tlwsys  be  interpreted  as  " wide-aease  stationary". 


specialized  subclass  of  the  class  claimed*  Later  in  the  same  year,  Sondhi  and 
Higgins  [ll,1959]  presented  a  solution  to  a  modified  form  of  the  shaping  filter 
problem  requiring  the  use  of  several  white  noise  sources*  Nothing  more  will  be 
said  of  this  work  because  interest  in  this  dissertation  is  restricted  to  the 
use  of  one  white  noise  input  process*  Finally,  Leonov  [12,1960]  presented  a 
rather  nice  mathematical  solution  to  the  shaping  filter  problem  for  continuoua 
parameter  processes  (both  stationary  and  nonstationary)  in  terms  of  expansions 
in  orthogonal  functions* 

In  the  following  paragraphs  a  precise  critical  summary  of  the  above 
mentioned  work  will  be  given* 

1*2.1  Summary  of  Previous  Work  for  stationary  Processes 

Since  no  useful  purpose  would  be  served  by  it  insofar  as  this  dissertation 
is  concerned,  no  attempt  will  be  made  to  summarize  the  previous  work  for  sta¬ 
tionary  processes  individually  as  cited  in  the  chronological  sketch*  Rather,  an 
over-all  summary  will  be  given,  the  details  of  which  can  be  found  in  the  books  by 
Doob  [ 1 3 , pp .  527  -  559  ,  569-590]  and  Grenander  and  Rosenblatt  [ 14 , pp .  6 5- 8 2] •  Only 
the  continuous  case  is  considered  and,  naturally,  T  =  ( -oo,  oo). 

If  the  stochastic  process  (Fit)}*  is  stationary  and  continuoua  in  the  mean, 
then  it  has  the  spectral  representation** 

CD 

Y(t)  =  f  ei2nktdZ(\)  (1*1) 

-  00 

where  the  process  (Z(\)}  has  orthogonal  increments  and  E\dZ{h)\2  =  dF(X)*  Fik) 
is  called  the  spectral  distribution  function  of  {F(t)}  and 

0D 

rrr(r)  =  £  m)  ru  +  t)  =  /  «ifwXr  </m)  u-2) 

-  00 

Furthermore,  F(M  is  nondecreasing  and  since 

00 

/  </m>  =  rff(oi  <  ®  d-3) 


*{f(f)}  is  to  be  real  valued* 

**For  a  definition  of  all  stochaetic  integrals  herein  sea  Reference  13,  p.  426* 


4 


Fik)  is  also  of  bounded  variation.  Hence,  Fik)  can  be  decomposed  into  the  sum 
of  three  nondecreasing  functions 

Fik)  =  F,  (k)  +  F2  ik)  +  F3(^)  (1-4) 

where  Fj(\)  is  the  jump  function  part  of  Fik),  F2(X)  is  the  absolutely  contin¬ 
uous  part  of  F(\),  and  F 3 ( X )  is  the  continuous  singular  part  of  Fik).  This 
decomposition  of  Fik)  corresponds  to  a  decomposition  of  {F(t)}  into  three 
mutually  orthogonal  processes  { Y 2i  t )},  and  {K  3 ( t ) }  with  spectral  dis¬ 

tribution  functions  F^ik),  F 2 ( X )  and  F 3 ( X )  respectively. 

If  {Tit)}  is  applied  to  the  input  of  a  linear  sysrem  whose  frequency 
response  function,*  G(X),  satisfies  the  condition 

00 

/  |GU)|2  dFik)  <  03  (1-5) 

-  00 

then  the  system  output,  0f(t)},  will  be  a  continuous  in  the  mean,  stationary 
process  whose  spectral  distribution  function,  Fj(X),  is  given  by 

X 

Fx(\)  =  /  |GU) | 2  dFik)  (1-6) 

"  00 


It  should  be  noted  that  G ( X )  is  not,  in  general,  required  to  be  in  Lz  (i.e., 

k 

it  is  not  required  that  J  | G ( X ) |  dk  <  go).  From  (1-6)  it  follows  that  Fj(X) 

-  00 

is  absolutely  continuous  if  Fik)  is.  If  Fik)  is  absolutely  continuous  and  if 
|/(X)|2  r  FMX),  then  (1-1)  can  be  replaced  by 


00 

Yit)  =  J  ei2nKt  fik)  dZik) 
-  00 


(1-7) 


«-V  'V 

where  {Zik)}  has  orthogonal  increments  and  E\dZik)\z  =  dk.  On  the  other  hand 
suppose  { K ( t ) }  is  generated  from  a  process  {K ( t  ) }  according  to  the  equation 


00 

Yit  )  =  /  Wir)  dVit  -  r) 

-  00 


where  { V  ( t  ) }  has  orthogonal  increments  with  E|dK(t)|2 
<  cu  .  Then 


(1-8) 

dt  and  /*  |(T(r)|  2dr 


00 

rff ( t , ,  t2)  =  f  iru,  -  e)  tu2  -  e)  d$ 


(1-9) 


Qik)  is  Doob’t  gain  function  C ( X ) . 


s 


From  (1-9)  it  follows  that  (F(t)}  is  stationary  and  continuous  in  the  mean. 
Furthermore 

k  9D  CD 

F(K)  =f  |G(  A.)  | 2  dk  <  J  |G(\)|2  dk  =  f  ||T(t)|2  dr  <  ®  (1-10) 

“  ®  “09  “09 

where  G(\)  is  the  Fourier  Transform  of  W(t)  and,  hence,  Fik)  is  absolutely 
continuous  and  F*  (k)  =  |G(A)|2.  Formally  considering  the  increments  cf  {F(t)} 
to  be  given  by 

V(t2)  -  K(tt)  =  J  iHt)  dt  (1-11) 

*  . 

where  {(/(t)}  is  a  white  noise  process,  (1-8)  represents  the  response  of  a 
linear  system  with  weighting  function,  IHt),  to  a  white  noise  input  process 
and  (1-9)  and  (1-10)  represent  well  known  results  which  are  usually  obtained 
in  a  less  rigorous  way  by  engineers.* 


As  a  consequence  of  the  above  results,  a  simple  necessary  and  sufficient 
condition  for  the  existence  of  a  solution  to  the  shaping  filter  problem  for 
continuous  in  the  mean,  stationary  processes  can  be  stated,  providing  the 
requirement  of  physical  realizability  is  waived.  They  are:  If  Fik)  is  the 
spectral  distribution  function  corresponding  to  the  given  covariance  function, 
rir),  then  a  process,  (F(t)},  whose  covariance  function  r^(r)  satisfies  the 
equality  =  F(r)  can  be  generated  from  a  white  noise  process  by  means 

of  a  linear  system  if  and  only  if  Fik)  is  absolutely  continuous.  Moreover, 
any  linear  system  whose  frequency  response  function,  G(\),  satisfies  the 
equal i  ty  |G(A) |2  =  F*  ik)  almost  everywhere  can  be  used  to  generate  such  a 
process,  (K(t)}.  More  generally,  even  if  Fik)  is  not  absolutely  continuous, 
the  above  still  applies  to  the  absolutely  continuous  part  of  Fik);  i.e.,  to 
F^ik)  in  the  decomposition  given  above.  If  the  requirement  of  physical  reali¬ 
zability  is  not  waived,  then  the  above  condition  must  be  strengthened  somewhat. 
Many  years  ago,  Paley  and  Wiener  [  1 S  •  p.16,  Theore*  XIl]  showed  that 

99 

if  J  |GU)  |  2  dk  <  cd  ,  where  G(\)  is  the  frequency  response  function  of  a 

“  99 

linear  system,  then  the  system  is  physically  realizable  if  and  only  if** 

/  dk  <  a>  (1-12) 

-  ®  1  +  k2 

In  view  of  the  required  equality  |G(A)|2  -  F*  ik)  i.e.,  physical  realizability 
of  the  linear  system  (the  shaping  filter)  requires  the  additional  condition 

*Tb«  usual  engineering  procedure  could  perhaps  b«  rigoriaad  at  the  expense  of  iatroduciag 
gaitaraliiad  linear  functionals*  # 

**The  a  a  aimed  continuity  of  Hr)  guarantees  that  J  |<?(A)|2  dk  <  •  . 

-  99 
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/  IWfMMl  a< 

1  +  k2 


(1-13) 


i.e.*  for  physical  realizability  of  the  shaping  filter,  F(k)  must  be  absolutely 
continuous  and  satisfy  (1-13).  Since  the  above  conditions  depended  only  on  the 
magnitude  of  G(\),  it  is  clear  that  these  conditions  do  not  uniquely  determine 
the  shaping  filter.  A  desirable*  way  of  rendering  the  shaping  filter  essentially 
unique  (to  determine  the  weighting  function  uniquely  except  on  a  set  of  Lebesque 
measure  zero)  is  to  require  it  to  be  a  minimum  phase  filter;  i.e.,  to  require 
that  G(\)  ^  0  for  Im  k  <  0*  Such  a  G(A.)  is  given  by  the  (loss-phase)  integral 


G(k) 


f  (1  +  log  F*  (oj) 

(\-u)(l  +  of)  . 


(1-14) 


Except  for  some  brief  comments  relative  to  some  special  cases  of  stationary 
processes  which  appear  at  appropriate  places  throughout  the  remainder  of  this 
dissertation,  this  concludes  the  summary  of  previous  work  for  stationary  processes. 
Clearly,  for  continuous  in  the  mean,  stationary  processes  and  T  =  (-  oof  oo)  the 
shaping  filter  problem  had  been  resolved  in  rather  definitive  terms  prior  to 
1950.** 


1.2.2  Summary  of  the  Work  of  Darlington 

In  his  paper  [9],  Darlington  presents  a  generalization  of  Bode  and  Shannon’s 
results  [8]  so  as  to  include  nonstationary  processes.  In  his  generalization, 
as  in  Bode  and  Shannon’s  original  procedure,  the  central  problem,  of  course,  is 
that  of  resolving  the  shaping  filter  problem.  As  will  be  clear  from  the  summary 
to  follow,  the  shaping  filter  problem  considered  by  Darlington  is  somewhat  dif¬ 
ferent  from  (but  related  to)  that  described  in  Section  1.1. 

After  some  preliminary  remarks  on  the  Bode-Shannon  model  and  its  use  of 
shaping  filters,  Darlington  turns  his  attention  to  the  shaping  filter  problem, 
proceeding  as  follows.  If  W(t,r)  denotes  the  weighting  function  of  a  linear 
system  then  the  covariance  function  T( t  , ,  t 2)  of  the  output  of  the  system,  when 
the  input  is  a  white  noise  process,  is  given,  when  it  exists,  by  the  expression 

OD 

n«„  t2>  =  /  m,,  r)  nt2,r)  dr  (i-i5) 

-  00 

Note  that,  since  the  lower  limit  of  the  integral  is  -  a ;  ,  it  has  tacitly  been 
assumed  that  the  white  noise  input  has  been  applied  to  the  system  continuously 

• 

The  inverse  filter  corresponding  to  s  physically  realisable,  minisiuai  phase  filter  is  physically 
realisable  and  stable.  This  is  iwportant  for  applications  to  linear,  lesat- squares  filtering 
and  prediction  theory. 

**TK.  shaping  filter  probl*.  is  sppsrsntly  still  unresolved  for  ststionsry  processes  ehich 
not  continuous  in  the  seen. 


are 


throughout  the  infinite  past.  If  the  system  is  physically  realizable*  then 
H^(l,  r)  =0  for  r  >  t  and  the  upper  limit  of  the  integral  in  (1-15)  can  be 
replaced  by  min[t,,  t2]»  Letting  Wa(t  ,  r)  denote  the  weighting  function  of  the 
adjoint  system,  Vta[t9r)  =  I Hr,t)%  and  r(  1 1  *  t2)  can  be  expressed  in  the  equiva¬ 
lent  form 

OD 

nt,,  t2)  =  /  H'(t1,T)r»(T,  t2)dT  (1-16) 

-  00 

Since  (1-16)  expresses  T( t ( ,  t2)  as  the  convolution  of  two  weighting  functions, 
n*1§  t2)  can  be  interpreted  as  the  weighting  function  of  the  nonphysical ly 
realizable  (self-adjoint)  system  composed  of  the  original  system  in  cascade  with 
its  corresponding  adjoint  system. 

Because  Darlington  was  unable  to  find  a  suitable  nonstationary  analog  of 
the  loss-phase  integral  (i,e.,  Equation  1-14)  for  solving  the  nonstationary 
shaping  filter  problem,  he  restricts  his  attention  to  systems  which  are  com¬ 
pletely  characterized  by  finite-order,  linear  differential  equations  and  seeks 
an  analog,  in  terms  of  operations  with  differential  equations,  of  the  usual 
procedure  of  factorization  of  the  rational  spectral  density  function  in  the 
corresponding  stationary  case. 

When  the  system  is  completely  characterized  by  a  finite-order  linear 
differential  equation,*  then  its  response  V  is  related  to  its  excitation  E  by 
an  expression  of  the  form 


B(p ,  t)  V(t)  =  Hit)  Aip ,  t)  Eit) 


(1-17) 


where  B(pt  t)  and  A(p,  t)  are  polynomials  in  p  with  time-varying  coefficients 


where  p  =  jy  ;  i . e. , 


«(/>,  O  =  Pn  +  bn  -  *)pn  “  '  + 

Up,  t)  =  p9  +  an  _  t{t)pm  ~  + 


+  6„(  t) 


♦  «0(t) 


( 1  - ] 8) 


and  flit)  is  a  time-varying  scale  factor.  Any  set  of  n  linearly  independent 
solutions,  say  (/^(t),  i  =  1,  •  •  .9  n,  of 


Bip .  t)  Vit)  =  0 


(1-19) 


Stt  Chapters  2  end  3  of  this  dissertation  for  details  of  tost  of  the  following  discussion. 


form  a  set  of  basis  functions  (6/'s)  for  Bip,  t)  and  for  the  system.  Similarly, 
any  set  of  m  linearly  independent  solutions  of 

A(Pt  t)  E(t)  =  0  (1-20) 

form  a  set  of  basis  functions  for  A(p ,  t)  and  are  called  the  zero  response  func¬ 
tions  (zr/'s)  of  the  system.  If  the  system  is  also  time- invariant  (stationary), 
then  the  6/'s  and  zr/'s  are  exponent i al s ,* e  *  where  the  s  are  the  familiar 
poles  and  zero  of  the  system  transfer  function.  The  6/'s  and  zr/'s  of  non¬ 
stationary  systems  play  analogous,  equally  important  roles  even  when  they  cannot 
be  represented  by  simple  coefficients  likeS  . 

O’ 

When  two  systems  are  cascaded  where  both  are  completely  characterized  by 
a  finite-order  differential  equation,  then  the  over-all  system  is  completely 
characterized  by  a  finite-order  differential  equation  corresponding  to  the 
product  of  the  differential  equations  of  the  two  given  systems.  In  terms  of 
operators,  the  product  may  be  represented  by  ** 

B]  V }  =  tf,  A  j  Et  B2V  =  H2  A2  V}  ,  BV  =  HAE  (1-21) 

where  BV  -  HAE  is  the  differential  equation  of  the  over-all  system.  As 
Darlington  indicates,  the  operators  B,  A,  and  H  can  be  determined  from  By  ,  B2, 
Ay,  A 2,  Hy,  and  H2  by  means  of  derivative  and  algebraic  operations.  This 
corresponds  formally  to  the  convolution  of  the  weighting  functions  of  the  two 
given  systems.  Similarly,  corresponding  formally  to  the  sum  of  weighting  func¬ 
tions  is  a  suitably  defined  sum  of  their  corresponding  differential  equations 
represented  by 


B,  K,  =  Hx  At  E  .  ti2  V2  =  H2  A2E 

V  =  K,  +  V2  ,  BV  ~  HAE 


(1-22) 


The  operators  B  and  A  and  the  scale  factor  H  can  also  be  determined  from  B , , 

B  2  i  A]t  A  2,  II  y ,  and  II 2  by  means  of  derivative  and  algebraic  operations.  Further, 
the  bf’s  of  B  are  those  of  By  plus  those  of  B2 ,  but  the  bf's  of  A  (the  zr/'s  of 
the  sum  system)  are  not  related  to  those  of  Ay  and  A 2  in  any  simple  way. 


Corresponding  to  the  system  characterized  by  (1-17)  is  its  related  ad¬ 
joint  system  which  is  completely  characterized  by  the  adjoint  differential 
equat ion 


Baip ,  t)  f(t)  =  ±  Hit)  Aa(p,  t)  Eit)  (1-23) 


Or  linear  combinations  of  them. 

Suppressing  the  arguments  for  convenience  of  notation. 


o 


corresponding  to  (1-17),  the  operators  Ba(p,  t)  and  4 a(p,  t)  being  easily 
determined  from  B(p,  t )  and  A(p ,  t).  When  the  system  is  physically  realizable, 
the  weighting  function  corresponding  to  (1-17)  can  be  expressed  in  the  form* 


/  y  Ui(t) 

W{t’  T)  =\  4",  T77TFT  Jiir) 


,  t  >  T 
»  *  <  T 


(1-24) 


and  that  corresponding  to  (1-23);  i*e*,  that  of  the  nonphysical ly  realizable 
adjoint  system;  in  the  form 


Wa(t,T)  = 


n 

y 

1/<M 

L 

i  ■ 

\  TTTiT  J<  ' 

.  * 

<  r 

(i- 

•25) 

0 

.  t 

>  r 

(1- 

23)  corresponds 

to  the 

convolution  of  lf(t, 

r) 

and 

is 

written  as 

- 

±  H*(t)  Arip,  t) 

i  E{t) 

(1- 

•26) 

From  the  discussion  following  (1-16),  it  is  clear  that  the  weighting  function 
of  the  system  characterized  by  (1-26)  is  H 1 1 ,  1 2 ) ,  which,  from  (1-16),  (1-24) 
and  (1-25),  can  be  written  in  the  form 

r  n 


l 


T77T77T 


>  t 


2 


i 


n  t 


i  * 


(1-27) 


The  symmetry  of  T(tt,  1 2)  expresses  the  fact  that  (1-26)  is  a  self-adjoint 
equation* 


With  this  background,  Darlington  takes  up  the  shaping  filter  problem  as 
encountered  in  the  Bode-Shannon  model,  assuming  that  the  signal ,  S(t),  and 
noise,  N(t),  are  generated  from  uncorrelated  white  noise  sources  by  means  of 
physically  realizable  systems  characterized  by  finite-order,  linear  differential 
equations*  If  the  6/'s  and  ir/'s  of  the  systems  are  known,  then  the  weighting 
functions  of  the  systems,  and  Wj,(t,r)t  are  easily  determined  and 

*Asauwing  order  n  of  B  >  order  a  of  4. 
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PyU^tj)  and  ( tj,  tt)  found  from  (1-16).  Also,  differential  equations  of  the 
form  in  (1-26)  can  be  found  for  both  r^(t|#t2)  and  by  forming  the 

product  of  the  corresponding  differential  equations  and  their  adjoints  as  de¬ 
scribed  above  (even  if  the  bf's  and  irf't  of  the  generating  systems  are  unknown). 
If  F  =  S  +  Af.  then  =  F^<t  |#  t2)  +  r^(t|#*2^  and  •  differential  equation 

of  the  form  in  (1-26)  whose  corresponding  weighting  function  is  c*n  be 

found  from  *2)  itself  or  by  summing  the  differential  equations  corresponding 

to  rsitvt2)  and  Pgit^$t2)  in  case  * 8  unknown.  In  this  way.  there  is 

determined 


B^(p.t)  V(t)  =  i  H^it)  A^(p,t)  Eit) 

Brt(p,t)  Vit)  =  ±  H)it)  A^[p,t)  Bit) 

Brf(p,t)  V(t)  =  i  HjU)  Arf(p,t)  E[t) 


(1-28) 


P  P  P 

The  b/'s  of  Bp(p,t)  are  those  of  Blsiptt)  and  fij(p.t)  and  are  even  in  number,  one- 

half  of  them  being  the  bf*s  of  the  systems  used  to  generate  *S(t)  and  N(t)  and  the 

other  half  being  the  bf's  of  the  corresponding  nonphysi cal 1 y  realizable  adjoint 

P 

systems.  On  the  other  hand,  the  6/'*  of  Aj(p. *)»  again  even  in  number,  are  not 

P  'p 

simply  related  to  the  bf*s  of  A^p.t)  and  Aj(p.t)  and  must  be  found  as  the  solu¬ 
tions  as 


AVf[p,t)  E(  t)  =  0 


(1-29) 


This  corresponds  to  the  calculation  of  the  zeros  of  the  rational  signal-plus- 
noise  spectral  density  function  in  the  stationary  caae  in  which  the  spectra) 
densities  of  5  and  N  are  added  (corresponding  to  forming  the  sum  of  the  dif¬ 
ferential  equations  for  t^)  and  to  get  the  spectral  density  of 

F .  The  addition  retains  the  poles  but  the  zeros  must  be  calculated  as  the  zeros 
of  a  polynomial  (corresponding  to  finding  the  solutions  of  Equation  1-29). 


Now  the  shaping  filter  problem,  as  considered  by  Darlington,  is  that  of 
finding  a  weighting  function  VI  pi  t,r)  such  that  the  systems  corresponding  to 
both  it  and  its  inverse  are  physically  realizable  and  behave  suitably  as  r  -•  ® 
for  al 1  t  and  such  that 


'VV  t2)  =  f 


W  (t  ,  r )  *a.(T,t,)dT 

r  \  r  2 


(1-30) 


To  do  thi.  he  firat  find,  the  bf‘t  of  B^(p,t)  end  >(C(p,  t)  from  the  known  bf't 

P  P  ¥  W 

of  Bls(ptt)  and  fij(p,t)  and  by  solving  (1-29).  The  problem  then  is  to  assign 

one-half  of  them  to  Wpit,r)  and  the  remaining  half  to  so  that  the  re¬ 

quirements  demanded  of  Vtpittr)  as  stated  above  are  met.  if  possible.  Darlington 
shows  that  this  is  possible  and  shows  how  to  do  it  providing  the  coefficients  of 


11 


the  differential  equations  characterizing  the  systems  used  to  generate  5  and  N 
are  regular  at  t  =  oo,  are  periodic,  or  are  of  moderate  variation*  In  these 
cases  the  bf9s  either  become  exponentials  as  t  "*  i  oo,  are  exponentials  multi¬ 
plied  by  periodic  coefficients,  or  are  dominated  by  exponentials  as  t  ^  i  oo 
and  those  bf9s  associated  with  exponentials  eSat  where  <  0  are  assigned 

to  Wf(t,n)  just  as  in  the  stationary  case,  the  H^(t,r)  thereby  obtained  having 
the  required  properties. 

Before  proceeding  to  a  summary  of  Batkov’ s  work,  it  should  be  noted  that 
in  Darlington's  work  it  was  assumed  that  T  =  ( -  oo,  oo)  and  that  was 

known  to  be  the  sum  of  two  processes  which  were  generated  from  uncorrelated 
white  noise  sources  by  physically  realizable  systems.  Further,  no  terms  due 
to  initial  conditions  are  present  in  or  r^(tpt2)  because  of  the 

assumption  of  stability  of  the  5  and  N  shaping  filters  and  the  choice  of  inter¬ 
val  T. 


1.2.3  Summary  of  the  fork  of  Batkov* 

In  his  paper  [lo],  Batkov,  like  Darlington  but  from  a  somewhat  different 
point  of  view,  also  studies  the  properties  of  weighting  functions  for  physically 
realizable  linear  systems  characterized  by  finite-order,  linear  differential 
equations  and  the  properties  of  the  covariance  functions  of  the  stochastic 
processes  at  their  outputs  when  their  inputs  are  white  noise  processes  with  the 
systems  starting  from  rest;  i.e*,  the  initial  conditions  are  zero.  On  the  basis 
of  this  study,  he  presents  three  methods  for  solving  the  shaping  filter  problem 
for  this  class  of  nonstationary  stochastic  processes.  The  first  of  these  methods 
is,  in  essence,  that  of  Darlington  but  is  not  quite  as  fully  developed  as 
Darlington's.  The  second  is  an  algebraic  method  using  the  discontinuities  of 
the  partial  derivatives  of  )  with  respect  to  tf  along  the  line  t,  =  1 2; 

but,  as  mentioned  earlier,  it  is  much  more  restricted  in  application  than  claimed. 
The  third  uses  a  method  due  to  Levy  [  1 5 ]  for  solving  a  certain  type  of  nonlinear 
Volterra  integral  equation  of  the  second  kind  by  resolving  kernels,  the  resolv¬ 
ing  kernals  being  solutions  of  linear  Fredholm  integral  equations  of  the  second 
kind.  These  methods  and  the  results  leading  up  to  them  are  briefly  summarized 
be  1 ow, 


Specifically,  Batkov  considers  a  physically  realizable  system  characterized 
by  the  differential  equation** 


L(p,t)  X(t)  =  Mlp,t)  Yit) 


(1-31) 


Baikov’s  paper  contains  several  functions  and  operators  with  arguaienta  incorrect. 

Many  of  the  results  stated  above  are  derived  in  detail  in  Chapters  2  and  3  of  this 
dissertation. 
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where 


Up,  t) 


M(p,t ) 


Z 


i«o 


«,(*) 


dt* 


R 

Z  6  j  ( t ) 
j-o  J 


di 

dti 


d 


p  =  JF  *  ■  <  " 


(1-32) 


and  the  a^(t)  and  6^(t)  have  n-derivatives  in  the  region  of  interest*  The 
weighting  function  G(t,r)  of  a  system  characterized  by  (1-31)  with  M(p,t)  =  1 
ia  the  solution  of 


Up,  t)  G(t,r)  =  Bit  -  r)  (1-33) 

with  zero  initial  conditions*  The  weighting  function,  Ga(t,r)  of  the  corres¬ 
ponding  adjoint  system  is  the  solution  of 

La(p,t)  C°( t ,r)  =  -  S(t  -  r)*  (1-34) 

with  zero  initial  conditions  and  Ga(t,r)  =  G(t, e).  When  M(p,t)  is  not  a  con¬ 
stant,  the  weighting  function  I Ht,r)  of  the  system  is  the  solution  of 

L(p,t)  H(t,r)  =  M(p,t)  Bit  -  r)  (1-35) 

with  zero  initial  conditions  and  can  be  obtained  from  Git,r)  by  the  expression 

I Ht.r)  r  Maip,T )  Git,r)  (1-36) 

The  weighting  function  Wa(t,r)  of  the  corresponding  adjoint  system  is  the  solu¬ 
tion  of 


Laip,t )  I f°(t,r)  =  -  Maip,t)  Bit  -  r)  (1-37) 

witli  zero  initial  conditions  and  Wait,r)  =  Wir,t)m  As  a  function  of  r, 

Wit,r)  satisfies  the  equation. 


rtlp.T) 

H'lt.r)  =  Qlp,r) 

sit 

-  r) 

(1-38) 

where 

n 

Rip.r) 

=  ^  r  i(r) 

“\ 

t  *  o  dr1 

► 

(1-39) 

Qip.r) 

=  fj  ,.(r) 

j  mo  J  drj 

> 

The  superscript  a  denotes  the  adjoint  differential  operator. 
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the  coefficients  r^(r)  and  g^lr)  being  determined  from  the  coefficients  a^r) 
and  bj(r)  algebraically.  Now  ^(t,r)  also  is  the  solution  of 

Hp,t)  H(t,r)  =  0  (1-40) 


with  the  initial  conditions 


=  0  ;  i  =  0*  , , , ,  n-*-2 


t  i  r 


3iIT(e,r)/3ti 

=  l/aB(r)|  2  (-1)*"-*  (  *  )  (t) 

t  1  r  L  \  J  /  J 

(vku  h  . 


(1-41) 


n-;  -2 

V  Br  *(t,r) 


«-rr“  i 


r*n-a- t 


1 1  i  r 


'£ 


(-1  )* 


0 


j  —  0  *  • • * i  *• 

Again,  is  also  the  solution  of 

R(ptr )  ir(t,r)  =  0 

with  the  initial  (final!)  conditions 


hmn.  I  3r  *  ,T  T  t 


X  ^  (-1)*  ^  *>(,),  j  =  n-.-l,  ....  n  -  1 


2 

■*" 

^>0 

where 


an°  G(t,r)/3rn+*  I  =  l/a„(  t )  2  -■  -  j-”  |T  ,  t  * 

It  t  t  *■»*- '  dr* 

n 

X  2  (-1)""*-'  ('  +  k)  ,**♦*-')(!)  ;  *  =  . . .  ... 

i.-fc-*  v  *  / 


(1-42) 


(1-43) 


(1-44) 


i>0 


Moreover,  ^(t,r)  can  be  expressed  in  the  form 


14 


I r(t,r)  =J  ^  9i(t)  >3i (r) 

i«r 


;  t  >  r 


(1-45) 


;  t  <  t 


where  the  g^tt)  and  /^(r)  are  sets  of  basis  functions  for  L{p,t)  and  ft(p,r) 
respectively. 

Assuming  {F(t)}  is  a  white  noiee  process  applied  to  the  system  at  time 


and  that  the  system 

is  at  rest  at  tQ*  the  covariance 

function  T(  t , , 1 2) 

of  the 

output  process  can  be  expressed  in  the  form 

t. 

/  l(t,,T)  1 r(t2,r)  dr 

t  „ 

•  *  1 

>  <2 

n*,,t2)  =  < 

0 

(1-46) 

f  W{tt,r)  W(t2tr)  dr 

;  *i 

<  *2 

Applying  the  operator  L(p,t()  to  (1-46)  yields 

Lip.t,)  n t , , t 

O 

II 

N 

;  *, 

>  *2 

(1-47) 

Up.t,)  nt,,t 

2)  =  1 r(t2,t,) 

;  *. 

<  *2 

(1-48) 

Further*  r(t1#t2)  has  2n-2*-2  continuous  partial  derivatives  with  respect  to  t, 
and  t 2  and  for  k  >  2n-2«-l 


rfc(  <  2, 1 2 )  = 

**«♦•  i 

z 


BtT  *'  ’  **  BtT  *>*  *2 
V  r  i  raw,|iu„wi  \ 

I  l  3t*  l  J  X  T  *»/  *<  1  ** 


(1*49) 


Also*  from  (1*47)  and  (1-49)*  or  from  (1-45)*  it  follows  that 


nr,.!*) 


f  n 

l 
<•  1 

9i<* »> 

Pl(t2) 

;  t,  ><2 

91 

l 

V  {  ■  | 

*  2  > 

Pi<  ‘  1  > 

:  4>  <  «2 

(1-50) 


where  the  are  a  set  of  basis  functions  for  L(p,tt)  and  the  P  4  ( ^  |  ■ 

set  of  particular  solutions  of  (1-48). 


u 


The  first  method  suggested  by  Batkov  for  solving  the  shaping  filter  pro* 
blem  is  based  on  (1-48).  Given  the  g^t,),  the  a^t,)  and  G(t,r)  are  easily 
computed  algebraically  (see  Chapter  2).  Knowing  Hp,t})  and  G(t2ft,)  and  using 
(1-36),  (1-48)  becomes 

L(p,t] )  Ht  e2)  =  W(p,tt)  Ma(p,t  |)  G  ( 1 2#  1 1  ) ;  t,  <  t2  (1-51) 

The  remaining  step  is  to  find  the  product  operator  M(p,t^)  Ma{p,t t)  from  (1-51) 
and  then  decompose  it  into  its  adjoint  factors.  As  Batkov  points  out,  this 
decomposition  is  difficult  if  M{p,t)  contains  derivative  operators,  but  is  simple 
if  it  is  just  a  time-varying  scale  factor.  Note  the  similarity  of  factoring  the 
product  M[p,t})  Ma(p,t])  here  and  the  factoring  of  Alip,t)  in  Darlington’s  work. 
These  are,  in  essence,  the  same  problem. 


The  second  method  described  by  Batkov,  i.e.,  the  algebraic  method,  is 
based  on  (1-49)  rewritten  in  the  form 

j-l  i 


t) 


!»diL  (p  ut)-  i  r  i  (0 

^(t.r)  d /  *lZ221il2!liLi2d  I  \  +  /  i  X^itu.t) 
ayt  dti-' t  \  &*”*-*-'  'KmtJ  V  *  / 

dJ'1  X 

7F*  E7rnjT=t  J  ;  J  =  n-«-1 . n*1 


(1-52) 


From  (1-52),  relationships  between  the  partial  derivatives  of  lf(t,T)  with  respect 
to  t  and  r  for  t  =  r  can  be  found  recursively  and  expressing  them  in  terms  of  the 
known  a^(t)  and  unknown  b^[t)  lead  successively  to  *-equations  in  the  b^(  t )  and 
their  derivatives.  Batkov  claims  that  6m_^(t)  enters  the  equation  obtained  from 
(1-52)  for  j  =  n-*+fe“l  algebraically  in  terms  of  6w(t),  •  •  .,  anc^  their 

derivatives.  Now,  in  particular, 

6,(t>  =  ±  an(t) J  (-1 '  r2n.2,.,(t,t)  (1-53) 

However,  as  will  be  seen  later,  6a_|(t)  does  not  appear  in  (1-52)  for  J * n - a  and 
both  6  ,(t)  and  6  1  ^  (  t  )  and  also  b9_z(t)  appear  in  (1-52)  for  >  =n-a+l,  etc., 
for  J =n-«+l ,  .  .  .,  n-1 .  Hence,  the  claimed  recursive  algebraic  method  for 
finding  the  bm_k(t)  from  (1-52),  and  thereby  solving  the  shaping  filter  problem 
algebraically,  fails  (except,  of  course,  where  bj{t)  m  0  for  >  f  0). 


The  third  method,  described  by  Batkov  in  an  appendix  to  hia  paper,  will 
not  be  summarised  at  this  point  because  of  the  difficulty  of  solving  the  Fredholm 
integral  equation  for  the  resolvent  kernel  and  because  of  hia  restrictive  assump¬ 
tions  as  noted  in  the  following  remarks. 


16 


Before  proceeding,  it  should  be  noted  that  Bstkov  assumed  that  the 
system  started  from  rest  at  t0  where  T  =  (t0,oo)  and,  what  is  even  more 
important,  he  also  assumed  that  r(t|#t2)  is  the  covarisnce  function  of  a 
process  generated  from  a  white  noise  process  by  a  physically  realizable 
linear  system  characterized  by  a  finite-order  linear  differential  equation 
of  the  form  (1-31)  for  t  >  tQ . 

1.2.4  Summary  of  the  Work  of  Leonov 

As  noted  in  the  chronological  sketch,  Leonov  C 1 2]  has  obtained  a  rather 
nice  mathematical  solution  to  the  shaping  filter  problem,  and  also  to  the  cor¬ 
responding  inverse  shaping  filter  problem,  in  terms  of  expansions  in  orthogonal 
functions.  This  work  is  summarized  here  and  the  details  can,  of  course,  be 
found  in  Leonov's  paper. 

Leonov  formulates  the  shaping  filter  problem  as  follows.  Given  a  white 
noise  process  (Tit)}*  where  -  co  <  t  <  oo  [i.e.,  Tj  -  l~  co,oo))  and  a  nonstationary 
process  (Jf(t)}  where  0  <  t  <  T  [i.e.,  Tx  =  (0 ,D],  it  is  required  to  show  that* 
the  random  function  [a  sample  function  of  {£(*)}]  Jf(t)  can,  under  certain  con¬ 
ditions,  be  represented  in  the  form 

*(t)  =  AxY(t)  (1-54) 

where  the  (linear)  operator  Ax  is  defined  if  the  function  Jf(t)  is  given.**  The 
corresponding  inverse  problem  is  that  of  representing  Y(t)  in  the  form 

HO  =  A }'  XU)  (1-55) 

where  Ax]  is  the  operator  inverse  to  Ax.  Leonov  shows  that  this  can  be  done 
by  explicitly  constructing  a  suitable  Ax  and  a  suitable  AJ*  as  follows. 

As  is  well  known  [16]  a  random  function  Z(t),  T g  =  ( a,  b ) ,  can  be  repre¬ 
sented  as  a  series  (canonical  expansion) 

OD 

Zit)  =  %  Bi  **(«>•••  (1-56) 

i-0 

*It  is  always  assuaied  that  £  Y(t)  ■  0  for  all  lAiti  noise  procassaa  considered  herein. 

**Pugechev  [  16 # 17]  calls  (2-54)  the  integral  canonical  representation  of  I(t). 

*#*The  z^(t)  are  not  necessarily  orthogonal  and  <  a  <  b  <  »  . 
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where  the  B ^  are  random  variables  which  satisfy  the  conditions 

E  B{  Bj  =  8ij  /)j  (1-57) 

and  the  z^t)  are  some  regular  (nonrandom)  functions.  In  order  that  the  series 
in  (1-56)  converge  in  the  mean  to  Z(t),  it  is  necessary  and  sufficient  that  the 
series 

oo 

r,(t,,«2)  =  2  Di  z{(t,)  (1-58) 

converge  to  ^ ,  1 2  )  *n  usual  sense.  The  definition  of  convergence  in  the 

mean  is,  of  course,  only  meaningful  for  random  functions  with  finite  variances. 

Now  to  solve  the  problem,  it  is  necessary  to  represent  T(t)  by  a  series  of 
the  form  in  (1-56).  However,  since  T(t)  does  not  have  a  finite  variance,  con¬ 
vergence  in  the  mean  cannot  be  used  and  a  new  concept  of  convergence  must  be  intro 
duced.  Leonov  introduces  the  concept  of  weak  convergence  in  the  mean.*  A  sequenc 
of  random  functions  Vn(t)  is  said  to  converge  weakly  in  the  mean  to  the  random 
function  U(t)  if  the  integral 

f 

an(T)  =  /  Rit)  Un[t)  dt  (1-59) 

0 

has  a  limit  in  the  mean  square  sense  as  n  oo  for  any  sufficiently  smooth  random 
function  R(t);  i.e.,  for  any  R(t)  which  has  finite  variance,  is  continuous  in 
the  mean,  has  the  necessary  number  of  continuous  stochastic  derivatives,  and  whose 
covariance  function  ( t  j ,  1 2 )  satisfies  the  inequality  f  *  it $  t )  dt  <  a o  •  With 
this  definition  of  convergence,  Leonov  shows  that  K(t)  can  be  represented  in  the 
form 

oo 

Y(t)  =  2  C,  y,(t )  (1-60) 

<»1 

where  E  C j  =  Sij  and  the  y ^ ( t )  are  any  complete  (in  L 2)  set  of  orthonormal 
functions  over  ( -  00,00)  and  where  the  series  in  (1-60)  converges  weakly  in  the 
mean  to  the  white  noise  random  function  K(t). 

To  solve  the  basic  problem  is  now  fairly  easy.  The  Cj  in  (1-60)  are  de¬ 
fined  as  follows 


(1-61) 


This  is  clearly  analogous  to  the  ordinary  concept  of  weak  convergence  in  Hilbert  apace  [is]. 


I  s 


where  the  random  variables  are  the  coefficients  in  the  series  expansion  of  Jf(t) 


00 

XU)  =  E  F(  XAt) 

i-  i 

(1-62) 

and  =  E  V  *  • 

The  linear  operator  Ax  is  then  defined  as 

CD 

m 

where 

Aj  Y(t)  =  j  1 Vx(t,T)  r(r)  dr 

-  0D 

00 

(1-63) 

Wx(t,T)  =  L  Jd,  XAt)  y  t  (r) 

i  •  1 

(1-64) 

Then  from  (1-63) 

and  (1-64) 

a)  00 

C  n 

Ax  Y(t)  = 

J  Vx(t,T)  Y(r)  dT  =  l  Vi  X*  (  t  )  =  X  (  t  ) 

-  00  t «  1 

(1-65) 

where  the  integral  in  (1-63)  is  taken  in  the  mean* 

The  functions  I fj(t,r)  and  Y(t)  in  (1-65)  can  be  defined  in  infinitely 
many  ways  by  using  any  other  representation  of  Jf(t)  in  the  form  (1-62)  as  is 
shown  to  be  possible  in  [  1 6 ] •  However,  Leonov  shows  that  if  F( t )  is  so  chosen 
that  (1-65)  holds,  then  there  is  one  and  only  one  I i  e* ,  lfj(t,r)  is  unique* 

Finally,  the  inverse  problem  is  easily  solved  as  follows*  Let  Aj1  be 
defined  as 


where 


CD 

A}'  XU)  =  /  IT}‘U,t)  X(t)  dr 

-  CD 


H'}'U,t) 


i 


y<(t )  ai(T)/'S~Di 


and  the  a^lr)  are  chosen  so  that 


i 


oi( t)  Xjir)  dr 


(1-66) 


(1-67) 


(1-68) 


As  before,  the  >{(()  are  any  complete  (in  L z)  set  of  orthonormal  functions  over 
(-00, oo).  Then  from  (1-60),  (1-66),  (1-67),  and  (1-68)  it  follows  that 

f  oo 

=  /  If  Z 1  ( t ,  r )  X  ( r )  dr  =  Yj  y  i  ( * ) 

0  i-, 


ru) 


(1-69) 
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The  series  in  (1-69)  converges  weakly  in  the  mean  to  the  white  noise  random 
function  K(t)  as  noted  above. 

This  completes  the  summary  of  Leonov's  solution  to  the  shaping  filter 
problem  and  the  corresponding  inverse  shaping  filter  problem,  the  remainder 
of  his  paper  being  devoted  to  mathematical  niceties  and  applications**  It 
is  clear  that  the  weighting  function  for  the  shaping  filter  can  be  written 
down  immediately  in  series  form  once  the  A^(t)  and  D ^  for  the  canonical  ex¬ 
pansion  of  A’ ( t )  are  known.  In  his  book  [16]  Pugachev  presents  several  tech¬ 
niques  for  finding  the  first  n-terms  of  expansions  of  the  form  (1-62)  rather 
simply  and  which  avoid  having  to  determine  the  eigenvalues  and  eigenfunctions 
of  an  integral  equation  as  required  in  the  well  known  Karhunen-Loeve  Expansion 
Theorem.  However,  it  should  be  noted  that  Leonov's  solution  is  always  obtained 
in  the  form  of  an  infinite  series  and,  further,  there  is  no  guarantee  of 
physical  realizability  of  the  shaping  filter  or  its  inverse. 


Applications  will  be  discussed  in  later  sections* 


CHAPTER  2 


PROPERTIES  OF  WEIGHTING  FUNCTIONS 
FOR  A  CLASS  OF  LINEAR  SYSTEMS 


2,1  INTRODUCTION 

Because  of  their  importance  in  the  shaping  filter  aa  evidenced  by 
Chapter  1  and  thoae  to  follow,  for  purposes  of  detailed  review  and  for  later 
use  Us  available  reference  material,  this  chapter  is  devoted  to  the  investi¬ 
gation  of  the  properties  of  weighting  functions  (Green's  functions)  for  sys¬ 
tems  which  can  be  described  by  finite-order  ordinary  linear  differential 
equations  of  the  form 

n  n- 1 

E  aAt)  X(i)(t)  =  E  bAt)  ylikt)  ;  t  >  0  (2-1) 

i-o  Jm o  1 


where,  for  t  >  0,  on(t)  /  0  and  the  aj(t)  and  6^(t)  ere  continuous.  Since 
weighting  functions  and  their  derivatives  are,  in  general,  discontinuous  at 
t  £  r,  this  investigation  divides  naturally  into  two  parts:  properties  in 
the  regions  where  t  ^  r,  and  properties  of  the  discontinuities  at  t  =  r. 


2.2  PROPERTIES  IN  THE  REGIONS  WHERE  t  £  r 


It  is  well  known  that  the  general  solution  of  (2-1)  can  be  written  in 
the  form  [  19,  p.257] 

t  »- 1 

*(t)  =  /  dr  Glt.r)  bj(T)  y(J,(r)  + 

0  jm  0 

n-  I 

+  E  9i(t)  (2-2) 

i*  0 


where  the  q^(t)  are  n- independent  solutions  of  (2-3) 


£ 


0i  X(<)(t) 


0 


(2-3) 


for  which 


q^^(t)  =  S ^ j ;  i,j  =  0 . .  -  1  (2-4) 

t  A  o 


and  G(t,r)  is  the  weighting  function  (Green's  function)  for  (2-3),  If  G(t,r) 
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and  the  6^(t)  have  a  aufficient  number  of  derivativea  with  reapect  to  r,  then 
through  integration  by  parts,  [20,  p.189]  (2*2)  can  be  brought  into  the  form 

t  n“ ' 

X(t)  =  f  dr  W(t,T)  y(r)  +  Z  X{i)(0)qAt)  (2-5) 

o  i«0 


where 


I nt.r) 


(-l)J 


3^6j(r)  G(t,r) 

drJ 


(2-6) 


Here,  If ( t , r )  is  the  weighting  function  for  (2-1).  Its  properties  are  investigated 
below* 


Since  f/(t,r)  is  defined  in  terms  of  G(t,r)  as  shown  in  (2-6), 
gation  of  the  properties  of  Ift  t,r)  must  begin  with  an  investigation 
perties  of  G(t,r)0  By  definition  [19,  p.254],  the  weighting  function 
is  that  solution  G(t,r)  of  (2-3)  which  satisfies  the  condition 


any  inveati- 
of  the  pro- 
for  (2-3) 


G(t,r)  =0  ;  t  <  r 


lim  3*G( t , r) 
tir  dti 


=  Os 


lim  jC(  t ,r) 

tir  Btn-i 


n-2 


(2-7) 


Since  G(t,r)  is  a  solution  of  (2-3)  for  t  >  T,  in  this  region  it  can,  according 
to  the  theory  of  linear  differential  equations,  be  expressed  as  a  linear  combina¬ 
tion  of  the  Hence 


n-  1 

G(I,t)  =  E  a,  qAt ) 
{ ■  o 


t  >  T 


(2-8) 


where  the  a ^  are  chosen  so  as  to  satisfy  conditions  (2-7).  Putting  (2-8)  into 
(2-7)  and  writing  the  result  in  vector-matrix  form  there  results 


9o(t|  •  •  • 

• 

a 

ao 

• 

0 

• 

• 

• 

90("‘')‘t)  •  •  * 

• 

• 

• 

• 

• 

• 

l/an(r) 

(2-9) 


Or  any  otktr  i«t  of  n-linaarly  indapandant  solutions  of  (2-3)  . 
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Now  the  determinant  of  the  matrix  in  (2*9)  ia  the  Wronakian  of  the  g^(t)  and 
hence  doea  not  vanish  because  the  ^(t)  are  a  set  of  linearly  independent 
solutions  of  (2-3).  Therefore*  the  inverse  of  the  matrix  in  (2-9)  exists  and, 
hence*  the  are  given  by 


q0lr)  .  .  .  <jn.,(T) 

•  * 

-1 

0 

* 

.  . 

.  . 

Jfln",l«r)  .  .  . 

. 

. 

_l/an(T>_ 

(2-10) 


It  is  seen  from  (2-10)  that  the  depend  on  r  only*  which*  together  with  (2-8)* 
implies  that  G(t,r)  is  separable ;  i.e.*  that 


G(t,r)  =< 


n-  \ 


i*  0 


c^lv)  qi(t) 


t  >  T 


t  <  T 


(2-11) 


V 

Under  the  assumptions  made  up  to  now  regarding  the  a^t)*  it  follows  from  the 
existence  theorem  for  solutions  of  a  differential  equation  that  the  q ^ ( t )  have 
at  least  n-continuoua  derivatives  for  t  >  0*  and*  thus*  from  (2-10)  it  follows 
the  glj(t)  are  at  leaat  continuous  for  r  >  0.  More  generally*  if  the  o^^(t) 
are  continuous  t  >  0*  then  the  a^(r)  have  at  least  n-continuous  derivatives 
for  t  >  0.  To  show  this*  it  is  convenient  to  introduce  the  adjoint  differential 
equation  corresponding  to  (2-3);  namely* 


n 

i 


(-1)* 


to^t)  X(t)]U)  =  0 


(2-12) 


Since  the 
form 


are  assumed  to  be  continuous* 
n 

E  CtU)  X(<)(t)  =  0 

i  *  o 


(2-12)  can  be  rewritten  in  the 


(2-13) 


where  the  Ct(t)  are  continuoua  for  t  >  0.  Now,  if  Hit.r)  ia  uaed  to  denote  the 
weighting  function  for  (2-13),  then  (19,  p.2S6] 


M(t,r)  =  G(t, t ) 


(2-14) 


Furthermore,  W(t,r)  muat  aatiafy  (2-13)  for  all  t  >  r.  Hence,  uaing  (2-11) 
it  ia  found  that 


0 


t  >  T 


(2-15) 


it- 1  n 

^  q{[T)  Z  Cj(t)  aj^(t)  = 


Since  the  q^ir)  are  linearly  independent,  (2-15)  implies  that 


Z  CAt)  a[ikt)  =  0 

i  •  a  J  1 


(2-16) 


Thus,  the  a^(t)  are  solutions  of  (2-13)  and,  hence,  have  n-continuous  deriva¬ 
tives  by  the  existence  theorem* 

From  (2-16),  it  is  easily  shown  that 


Z  CAt)  a^lUt) 
*[nUt)  =  -  il°  _ _ _ 


(2-17) 


From  (2-17),  it  is  easily  concluded  that,  if  the  C^*^(t)  are  continuous  [i*e., 
the  +  are  continuous],  then  the  are  continuous*  Similarly, 

from  (2-3)  it  can  be  concluded  that  the  n  +  *Nt)  are  continuous  if  the  o^^(t) 
are  continuous.  This  concludes  the  study  of  G(t,r), 


Substituting  (2-11)  into  (2-6)  it  is  easily  seen  that  )f(t,r)  can  be 
written  in  the  form 


ITU,t)  = 


n  —  i 

Z  Z  ( —1 )  ^  [6  Ar)ai[r)]  ^  ; 

i«o  jm o  ; 


(2-18) 


:  t  <  r 


The  existence  of  the  derivatives  in  (2-18)  will  be  guaranteed  by  assuming 
continuity  of  the  and  6^.^(t)  for  t  >  0.  Carrying  out  the  indicated 

differentiation  and  nrking  the  obvious  definitions  for  the  /3j(r),  it  is  found 
that  H Mt(r)  can  be  written  in  the  form 


#(t,T)  = 


q{ < t  )fii(r) 


t  >  r 


;  t  <  r 


(2-19) 


Hence,  W(t,r)  is  separable ;  i*e*,  can  be  written  as  the  sum  of  products  of  a 

function  of  t  only  by  a  function  of  r  only*  Further,  the  functions  of  (  are 
a  set  of  linearly  independent  solutions  of  (2-3).  It  is  clear  that  continuity 
of  the  derivatives  of  the  q^(t)  and  implies  the  continuity  of  the  partial 

derivatives  of  l?(t#r)  at  any  t,r  except  t  -  r.  Examining  the  differentiability 
properties  of  the  /^(t)  it  is  found  that  the  are  continuous  for  r  >  0 


2  S 

if  the  +  and  are  continuous  for  t  >  0*  Incidental lyf  it 

is  also  clear  that  W(t,r)  formally  satisfies  (2-3)  except  at  t  =  r. 

2*  3  PROPERTIES  OF  THE  DISCONTINUITIES  OF  I T(t#r)  AND  ITS 
PARTIAL  DERIVATIVES  AT  t  =  r 

In  order  to  investigate  the  discontinuities  of  ^(t#r)  and  its  partial 
derivatives  at  t  =  r%  it  will  be  convenient  to  replace  (2-1)  by  a  particular 
set  of  n  first-order  equations.  If  the  at(t)  and  6j(t)  have  a  sufficient 
number  of  continuous  derivatives  for  t  >  0,  then  (2-1)  is  equivalent  to  the 
following  system  of  first-order  equations  (see  [20,  p.l9l]). 

X(t)  =  X ,U)  +  F0(t)  yit) 

jrj 1  *  ( * )  =  X2it)  +  F,(t)  y(  t ) 


(2-20) 

*!-!<*>  =  +  '.-!<«)  yit) 

Xa(t).. .-•„(«)  X,(t)  +  Fn(t)  yit) 

where  the  F^(t)  can  be  found  recursively  from  the  equation 

i-i  i-fc  \ 

F.l.l  =  -  S  S  (2-21) 

k-t  s»o 

In  (2*21)  use  has  been  made  of  the  fact  that  F0it)  -  6 n ( * )  ■  0  and,  hence,  all 
terms  involving  F0it)  have  been  omitted.  If  W^it,T)  is  used  to  denote  the 
weighting  function  coi responding  to  X*(t)  in  (2*20),  then  letting  yit)  =  S(t-T), 
(2-20)  yields 

iMt.r)  =  IT,  ( t,  r) 

IT,1  1  *0>(t.T)=  ir2(  t.r)  +  F,  ( t )  8  it  -  t) 

•  •  • 

.  .  •  (2-22) 


IT*;: ;o)(  t.T) 
»i''0,(t.r) 


ir»(t»T)  +  Fn-\{t)  &  (t  -  T) 

n 

-  S  «,_.(*)  ir4  ( * .  t  )  +  F„(t)  8(t  -  r) 
i«  i 


Integrating  (2-22)  with  respect  to  t  from  0  to  I  and  remembering  that 


for  t  <  r,  there  results  (for  r  >  0) 


i  * 

/  dcr  W(crtr)  =  /  daW^a.r) 

T  r 

t 

-  j  da  H'jlcr.r)  +  F,(r)  £/  ( t  -  r) 


it.u.t) 


(2-23) 


l^n-  l  ( l»  T> 
H,n  ( t »  t  ) 


» 

/  dcr  WHlar,r)  +  Fn-,(r)  Ul  t  -  r) 


f  da 


T* 

.  la. 


L*  a..,  (a)  Hf((o',r ) 


(2-24) 
+  Fjr)U(t-T) 


where  U(t  -  r)  is  the  ordinary  unit  step  function.  Making  use  of  (2-22)  and 
(2-23),  the  discontinuities  of  Yi(ttr)  and  its  partial  derivatives  can  be  evaluated 
fairly  easily.  In  fact  from  (2-22)  it  is  easily  established  that 


H'|i'o)(t,r)  =  ir<  +  |(t,T)  :  t  >  t,  i  =  0 . n  -  1 

n 

I r(n'0,(t,T)  =  -E  a.  At)  WAt.r)  :  t  >  T 

i-1 

If  is  defined  as  shown  in  (2-25) 


(2-24) 


*  1  T  3t  *  *ri 

then  from  (2-23)  and  (2-24)  it  is  clear  that 


(2-25) 


,r(  4 , 0 )  (T,r ) 

(jr(»,o)(T,r) 


ftt1(r) 

n 

-  E  a4-l(T)  Fx{t) 

i-l 


Also  from  (2-24)  it  is  clear  that 


»  =  0, 


n  -  1 


(2-26) 


t)  =  f[®;J,(*.T)  ;  t  >  r  i  1=1 . n  -  1  (2-27) 

n 

W{n’JHt,r)  -  -  Eot.,(t)  ir50';  Ut,r)  ;  t  >T 

By  virtue  of  (2-27),  the  evaluation  of  the  discontinuities  of  the 
at  t  -  t  reduces  to  the  evaluation  of  the  discontinuities  of  the  at 

t  =  r.  To  find  these,  use  is  made  of  (2-23).  Upon  performing  the  indicated 
differentiation,  it  is  found  that 


O  -T 


f  dcr  H'j  +  ^ho’.r)  -  Z  kMr,r)] 

7-  0  L  J 

♦  f\)Ut)  ;  t  >  r,  i  <  n 


ft 

t.r)  =  J  da 


j -» 

S 

*«0 


-  Z  a(-|  (a)  ITi  ^°>^(<7,r) 

t  ■  1 

-  Z  an_,(T)  0 »  *  }  ( r ,  i  -  '  -  *  > 


i-1 


+  F^j  Ut)  ;  t  >  r 

Taking  the  limit  as  t  J  r,  there  results 

j-' 


iri°**,(T,T] 


2 

*■0 

j- 1 


(y-1-*)  +  j )(T)  ;  i  <  n 


wl°‘iht,T)  =  -  Z  -  Z  oi. ,  (r)  H'i0‘*)(t,r)1  lim'mk) 

hm0  l  J 

+  F^jHr) 


(2-28) 


(2-29) 


Equations  (2-29)  can  be  used  to  find  the  values  of  I ff[op^(r,r)9  and,  hence,  those 
of  I recursively  in  terms  of  the  F^(r)  and  at(r)  and  their  derivatives. 


2.4  A  SYNTHESIS  PROBLEM 


This  chapter  concludes  with  the  solution  of  the  following  synthesis  problem; 
Given  W(t,T)  expressed  in  the  form  in  (2-19),  find  the  a  ^  (  t )  and  bj{t)  of  its 
corresponding  differential  equation  (2-1)* 

Since  the  q^(t)  in  (2-19)  are  solutions  of  (2*3),  it  is  clear  that  the 
following  equations  are  valid.  n 

Z  at(t)  =  0 

t  «0 

•  • 

(2-30) 

n  •  • 

Z  „<(<)  =  0 

i  »o 

Hegarding  the  q^(l)  as  known  and  the  a^lt)  as  unknown,  (2-30)  is  a  set  of  n- 
I  inear  equations  in  the  n  ♦  1  unknowns,  ai(()*  Since  it  can  be  assumed  without 
loss  of  generality  in  (2-1)  that  «n(f)  «  1,  (2-30)  can  be  solved  for  the  remain¬ 
ing  unknown  a^(t)  with  the  result 


(2-31) 


1 

a 

o 

_ 1 

q0(t)  •  •  • 

• 

. 

• 

= 

• 

• 

. 

an-l(t) 

?„-,(*>  •  •  •  lisT'ho 

-9{0nht) 


The  existence  of  the  inverse  matrix  in  (2-31)  is  guaranteed  by  the  fact  that  the 
4{(t)  are  linearly  independent  solutions  of  (2-3).  Equation  (2-31)  provides  the 
desired  relationship  for  determining  the  a^(t)  from  0(t,r).  This  result  is  given 
as  Theorem  6.2  in  [ 2 l]  for  an  arbitrary  set  of  n- line  rly  independent  functions 
q^(t)  providing  the  q^(t)  have  n- continuous  derivatives  on  the  region  of  interest.* 
Once  the  a^(0  have  been  determined,  the  6^(0  can  be  found  from  (2-21)  rewritten 
in  the  form 


Fk{s)(t)  (2-32) 


where  the  F^(t)  are  given  directly  in  terms  of  the  discontinuities  of  the  weight¬ 
ing  function  and  its  derivatives  with  respect  to  t  as  in  (2-26)» 


A  development  similar  to  that  given  in  Sections  2.2,  2.3,  and  2.4  has  also 
been  carried  out  by  Borskii  [22]. 


Linear  independence  of  the  <?{(£)  ia  equivalent  to  the  nonvaniahing  of  their  Wronakian* 


CHAPTER  3 


PROPERTIES  OF  THE  COVARIANCE  FUNCTIONS  OF 
A  CLASS  OF  STOCHASTIC  PROCESSES 


3.1  INTRODUCTION 

As  is  to  be  expected  and  as  evidenced  by  Chapter  1,  the  covariance  func¬ 
tions  of  the  class  of  stochastic  processes,  which  can  be  generated  by  paasing 
white  noise  processes  through  systems  charac teri zabl e  by  finite-order  ordinary 
linear  differential  equations  of  the  type  given  in  (2-1)  with  random  initial 
conditions,  have  many  properties  in  common  in  addition  to  that  of  being  non¬ 
negative-definite  [23, p.466]*  This  chapter  is  devoted  to  the  development  of 
some  of  the  more  interesting  and  useful  among  these  additional  properties  both 
for  the  purpose  of  detailed  review  and  later  use* 


3.2  SOME  GENERAL  PROPERTIES 


Let  it  be  assumed  that  a  given  system  can  be  characterized  by  a  differen¬ 
tial  equation  of  the  form  given  in  (2-1)  where  an(t)  *  1  and  the  and 

are  continuous  for  t  >  0.  Then  the  weighting  function,  F(t#r),  for  the 
systems  exists  in  the  form  (2-19).  Now,  letting  (y(t)}  be  a  stochastic  process 
with  covariance  function  and  A^**(0),  random  variables  with  covariances 

rij$  the  covariance  function,  the  stochastic  process  (JT(t)}  can 

be  determined  from  r^(t|#t2)  an<*  ^ij  accor<linS  to  the  relation  [20,p.22?].* 


;«  r  2 

r/x(t,,t2)  -  J  dTy  J  dT2W(tt,T ,)  i m  t2,T2)ry>(T,  ,r2) 

0  o 

1  n-i 

+  L  L  rtj  <*<(*,)  qj(t  2)  ;  t,  >  0,  t2  > 


(3-1) 


i  *  0  j  *  0 

When  y(t)  is  a  white  noise  process,  ^  t ,  f  2  )  =  S(t1  -  *2)  and  (3-1)  beconea 

/ 


/  dr  ir( t ,, r )  i r(t2,r)  +  ^  ^  rij  ?*(*,)  ; 


i* 0  j* 0 


rxjr(e,,t2)  ~  \ 


t.  >  t2  >  0 


f *»  nv>' 

J  dr  »U,,r)  *  l  L  r4j  9i(tt)  q  ( 1 2 ) ; 

^  0  i*o  j*o 


«,  >  t,  >  0  (3-2) 


*Hers  it  has  baan  aasuaiad  that  f[/li,(o)y(  t )]  *  0  in  i  and  t  >  0. 


JU 


If  the  expression  for  lf(t,T)  given  in  (2*19)  is  substituted  into  (3-2)  snd  the 
coefficients  of  the  qiit)  are  collected,  (3-2)  can  be  brought  into  the  form 


n-1  ’’Z 1  t 2 


(r)  +  r. 


^1/^*1*  *2*  - 


t,  >  t2  >  0 


y  f  r  f‘<  1 


(3-3) 


t,  >  l,  >  0 

It  will  be  convenient  to  denote  the  terms  in  the  square  brackets  in  (3-3)  by 
the  symbols  p^(t),  in  which  case  (3-3)  becomes 


rXI ^  *  I  *  *  2  ‘ 


2  oAt 


*?i  <  *  i  >  PA*  i 


;  t .  >  t,  >  0 


7*-  I 

S  Pjlt ,)  ;  1 2  >  t ,  >  0 


(3-4) 


From  (3-4)  two  important  general  properties  of  the  class  of  covariance  functions 
under  consideration  are  obvious*  First,  they  are  separable  in  the  sense  used  in 
the  preceding  section*  Second,  for  t \>t  2,  the  functions  of  t(,  i*e*,  the  q((tt), 
are  solutions  of  the  homogeneous  differential  equation  (2-3)*  In  addition,  if  the 

a(i+fe-i)(t)  anj 

are  continuous  for  t  >  0,  then  has  con¬ 

tinuous  partial  derivatives  of  order  k  in  regions  t,,  t2  >  0l  * ,  £  1 2*  This  is 
easily  established  from  the  differentiability  properties  of  the  q^(t)  and  /3j(t) 
discussed  in  Section  2*2* 


3.3  DISCONTINUITY  PROPERTIES 


The  discontinuities  of  the  partial  derivatives  of  at  tf  =  t2 

can  be  evaluated  in  terms  of  the  discontinuities  of  the  partial  derivatives  of 
I f(t,r)  at  t  *  r  by  making  use  of  (3-2)*  In  fact,  upon  differentiating  (3-!!) 
partially  with  respect  to  t1  (assuming,  of  course,  that  the  requisite  derivatives 
exist)  there  results 


3! 


tz 

/  drir,,'0,U,,T)  itUj.t)  + 


n- l  n-  l 


*  2 

i»0  jm 0 


*1  >  «2 


t, 

/  *rt«,*0»(«l.ri  ir(t2,T)  +  iru,.*,)  + 


«- i  »- i 


♦  I  l  Ftl  it"  "■>  Ij'1!1  *•  >  »t 

\  **o  Jn0 

Defining  the  diecontinuities  Jijit 2)  as  shown  in  (3-6) 


it  follows  from  (3-5)  that 


*  *2(*2'*2> 


(3-5) 


(3-6) 


(3-7) 


In  genera) 


J  2dTWli’°Htvr)  I r^‘°Htvr)  + 

0 

{,*  0  dt  J' 

♦  1  t  rK ,  i  *>  >  ‘a 


rix,'i,(,r  **>  =\ 


*-0  1*0 
*1 


f  drir(i'0,(  t,,T)  + 

°<-i 

♦  J  -lL  fir<i-*-''0,(t1,t,)  f^'0,U2.*,)]  + 

*-0  3t,k 

*  t  l  r,.  •'  '« »  *' 


and  hence 


h*0  1*0 


j.  ,<•,>  =(z'  "i-'*1 

i,}  *  Ijw  o  3*5  J 


(3-8) 


*  i  *  *  * 


{  £  -ii- .  tx)  (r‘^0,(*2,  *,)]}*,  *  *2  (3-9) 

v  h*o  3l j  J 


It  should  be  noted  that  no  initial  condition  terms  appear  in  (3-7)  or  (3-9); 
i.e.,  the  discontinuities  of  the  9  i  *  ( 1 t  $  1 2 )  depend  only  on  the 

If  the  t,r)  appearing  in  (3-9)  are  expressed  in  terms  of  the  aj*Mt) 

and  the  Fj^*{t)  as  developed  in  Section  2*3,  then  (3-9)  becomes  a  second  degree 
equation  in  the  F^lt)  and  their  derivatives.  This  form  for  the  J.  .(t2)  will  be 
examined  in  more  detail  in  a  later  section. 


3.4  A  FINAL  IDENTITY 


Equations  (3-10)  can  be  considered  as  a  set  of  simultaneous  nonlinear 
integral  equations  in  the  /^(r)  if  the  p^t),  <?j(0>  and  are  assumed 


n- 1  *  «- I 

P{(t)  =  E  qAt)  J  dT/3i(T)/3j  (r)  +  E  Tiy  qAt) 
j-o  J  o  j* o  J  J 


0,  .  .  .  ,  n  -  1 


(3-10) 


known.  These  integral  equations  can  be  converted  into  a  set  of  simultaneous 
second  degree  differential  equations  in  the  ( t  )  which  are  independent  of  the 


n-  1 


r.j.  To  see  this,  the  operator  £  flv(  t  )  d*/dt*  is  applied  to  both  sides  of 
*  J  k*  o  * 

(3-10),  [assuming,  of  course,  the  requisite  differential  properties  for  the 
p^(t)  and  with  the  result 


"  n- 1  n  f 

E  ofc(t)p|fc,(t)  =  E  E  «»U>  ~  [qAt)  J  drfiAr)/iAT)  +  r{<]  ; 

*i»0  j- 0  km  0  dtk  J  0  1  J 

I  =  0,  ....  n  -  1  (3-11) 

Making  use  of  the  fact  that  the  qj(t)  satisfy  (2-3)  it  is  clear  that  the  terms 
in  (3-11)  involving  the  FH  vanish  identically  in  t  for  all  i9j%  Expansion  of 
the  remaining  terms  in  (3-11)  according  to  the  rule  for  differentiation  of  pro¬ 
ducts  gives 


s 


n- \  n  k 

N '  S'  Y 

akit)  =  Lj  L  L  IJ]  x 


j*  0  fc*  0  ,f  *0 
t 


x  [  j  dr  /3i  (  t  )  />  j{r)] 


( k-l) 


l  —  0,  .  •  .  ,  n  •—  1 


(3-12) 


Now  (3-12)  can  be  rewritten  in  the  form 
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»-l  n  *- I 


Z  «*(*>  p{*Mt)  =  Z  Z  Z  }]  ak(t)  X 

**0  0  *»0  ^*0  I"1'/  ' 

n- 1  n 

x  +  Z  Z  ak{t)  q\kHt)  X 

3  J-0  *-0  3 

t 

x  [  /  dT^i(T)fij(T))  i  <  s  0,  1  (3-13) 

0 

Again  using  the  fact  that  the  <7  j  ( t )  are  solutions  of  (2-3),  it  is  clear  that 
the  second  term  on  the  right-hand  side  of  (3-13)  vanishes  identically  in  t 
for  every  value  of  i  and  j  •  Further  expansion  of  the  terms  [/^  ( t  )fij  ( t ))  *  1  ) 

in  (3-13)  yields 

n  n-1  n  1 


Z  «*(t)  =  Z  Z  Z  Z  ■{ 

*■0  jrn  0  *■  0  £*0  «»0  I* 


k-l-1 

m 


X  ak(t)  ; 

i=0,  0 


(3-14) 


The  above  identities  form  the  desired  set  of  second  degree  differential  equa¬ 
tions  in  the  J3 i(t).  They  are  obviously  equivalent  to  (3-10)  providing  a  proper 
set  of  initial  conditions  for  the  /3^  ( t )  and  their  derivatives  is  given* 


3*5  A  DISCUSSION  OF  BATKOV’  S  ERROR 


As  noted  in  Chapter  1,  the  second  method  described  by  Batkov  in  his 
paper  only  works  as  an  algebraic  method  for  a  much  more  restricted  class  of  co- 
variance  functions  than  claimed  by  Batkov*  Using  the  results  of  Sections  2*3 
and  3,3,  this  is  easily  demonstrated  aa  follows. 

Letting  n  >  3  and  examining  ^  **  f°un(i  from  (3-9)  that 

J0>l(t2)  =  (3-15) 

which,  by  (2-26)  and  (2-21)  yield* 

</<>.. <««>  =  '?(*!>  =  (3-16) 


Similarly,  from  (3-9) 
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',0,2U2)  =  W*,.*,)  W*,.*,) 

+  H'Uj.tj)  »(*,,  t2)<n  (3-17) 

which  by  (2-26),  (2-29),  and  (2-21)  become* 

Jo,2(t2)  =  Fi(t2)  f2(t2)  +  -F2U2)  +  f‘n(t2)  F,(t2)  + 

+  F, ( 1 2 )  F j 1  * ( 1 2 )  =  2F , ( t  2 )  F{ 1 » ( 1 2) 

=  26 n.,(t2)  6njj>U2)  (3-18) 

Note  that  <^0,2^2^  does  not  depend  on  6n_  2  ( 1 2 )  which  is  contrary  to  what  Batkov 
claims  [Batkov ’s  Pfc(ta,*a)  is  equal  to  Further,  from  (3-9) 

JQ 3(t2)  =  if(t2,t2)  ir<2*0,(t2,t2)  +  »r(0'  n( t2# t2)  nrt'-°)(t2>t2) 

+  If(t2,t2)  trll*0)(t2, t2)(n  +  ir(0*2)(t2,t2)  i r(t2,t2) 

+  2H'<0',)(t2,t2)  ir(t2,t2)("  +  ir(t2,t2)  if(t2<t2)(2»  (3-19) 

which  by  (2-26)  and  (2-29) 

J0,3(t2>  =  Fl(t2>  f3(t2>  +  ^f2(t2)  +  F  }{t  Ht  z)  Fz  <  t 

4  Fx(tz)  F^}Ut2)  4  -2F^l)U2)  4  F\z](tz) 

4  f3(e2)  F,U2)  4  2  -Fz(tz)  4  F\'](tz)  F\})itz) 

4  F,U2)  F}2)lt2) 

=  2F,U2)  F3(  1 2 )  -  F2U2)  _  F  { 1  M  1 2 )  Fz(tz) 

-  F<n(t2)  F9lt2)  4  F  j  n  ( t  2 ) 2  4  F,  ( 1 2  )  F|2,u2)  (3-20) 

Making  use  of  (2-21),  it  is  a  function  of  bn_3lt2h  ^n-2^2^  *tg  first 

derivative,  and  6n-1(t2)  and  its  first  and  second  derivatives.  Proceeding, 
it  can  be  shown  that  a  similar  situation  obtains  for  the  higher  order  jumps, 
,j(*2>*  This  clearly  demonstrates  Batkov *s  error  and  shows  that  from  the 
jumps  one  can  obtain  at  best  s  set  of  simultaneous,  nonhomogeneous ,  nonlinear 
differential  equations  in  the  F^((2)  or  bj(f2)*  independent  set  can  be 
obtained  from  the  jumps  ^  1 2 )  where  i  =  0,  .  ,  .  ,  n  -  1  for  example. 

However,  because  of  their  complexity  and,  hence,  lack  of  utility,  they  are 


neither  derived  nor  considered  herein.  Of  course,  when  6t(t2)  ■  0  for  0  <  i 

<  n  -  1,  then  JifAt2)  =  0  for  i.j  <  n  -  1  and  J__.  _(*,)  =  t,)1 

,2/1.  **  4 

“  "o' *2'  an"  Batkov’s  algebraic  method  works.  However*  the  first  method 

described  by  Batkov  also  works  for  this  special  case.  Note  that  it  has  been 

assumed  in  the  above  that  T( t ( ,  1 2 )  is  the  covariance  function  of  a  process 

obtained  from  a  white  noise  process  by  the  physically  realizable  characterized 

by  an  n ^  order  differential  equation  of  the  form  given  in  (2-1). 


CHAPTER  4 


EXACT  SOLUTION  OF  THE  SHAPING  FILTER  PROBLEM  FOR  SOME  SPECIAL  CASES 

4.1  INTRODUCTION 

When  the  function  introduced  in  Section  1.1  has  certain  special 

properties,  the  shaping  filter  problem  can  be  resolved  by  fairly  elementary 
methods.  These  special  cases  are  studied  in  this  chapter  and  the  methods  of 
solution  are  presented.  It  is  assumed  that  T  =  (0,  T)  and  that  H t ( , t ^ )  is 
separable  and  is  given  in  its  separable  form, 

4.2  THE  "SIMPLEST”  CASE 


Probably  the  simplest  case  is  where  is  of  the  foi 

n  n 

=  Y  Z >  q ^ ( t ,  )  q j( t z) 

i  •  1  j-1  J  J 


(4-1) 


where  it  can  be  assumed  without  loss  of  generality  that  the  are  linearly 

independent  on  T.  In  this  case  a  set  of  necessary  and  sufficient  conditions 
that  r(tj,t2)  be  a  covariance  function  is  that  the  matrix  be  symmetric 

and  nonnegat ive-def ini te.  To  prove  the  sufficiency  of  the  conditions  it  is 
observed  that  if  [r^]  is  symmetric,  then  so  is  r(t1#t2).  Further,  considering 
the  expression  for  arbitrary  z  ^ 

«  «  n  n  m  m 

Y  Zj  r(tLt')  z  =  Y  Z>  r\,  Y  Yqiitl)  qAto)  zk  Zo 

*»  I  l-\  *  t  k  +  i-l  j-1  *•!  /-I  J  +  * 


n  n 

=  t  t  r{j 

i-i  j- i  J 


yi  y) 


(4-2) 


m 

where  y ^  =  £  q^  <*\>  *k,  it  follows  that  r(t|#t2)  is  nonnegat ive-def ini te  if 

k*  i 

[ri;]  is.  This  establishes  the  sufficiency.  On  the  other  hand,  symmetry  of 
r(f|#t2)  clearly  implies  symmetry  of  [r^J.  Further,  since  the  are  linearly 

independent  on  T,  there  exist  at  least  n-values  of  t  €  T,  say  such  that  the  matrix 
Ui<  *  is  nonsingular,  for  if  not,  the  qi(t)  would  be  linearly  dependent  on  T. 
For  arbitrary  y^,  let  z k  be  defined  by 

m  =  Ui  <  «;>]■'  (4-.U 

where  [*]*  =  (i,  .  .  .  rB]  and  ly]r  =  [y,  .  .  .  yn).  Hence,  the  expression 


(4-4) 


0  O 


n  n  n  n 

E  E  ri}.  ,i,.  =  E  E  n ti  ,  ti i  z,  *•  >  o 

i«1  Jm ,  J  k>l  {.|  i  *  t 

holds  for  arbitrary  yiP  the  inequality  following  the  nonnegat i ve- def ini teness  of 
r(t1#t2).  This  establishes  the  necessity  of  the  conditions. 

Now  given  that  [rdj]  is  symmetric  and  nonnegat i ve- de f i ni te ,  the  process 
whose  sample  functions  are  of  the  form 

n 

X(t)  =  E  A{  q{(t)  (4-5) 

t  =  1 

where  the  A ^  are  random  variables  and  E  A iA j  =  T has  r(t)#t2)  for  its  covariance 
function.  If  the  qi{t)  have  continuous  nth  order  derivatives  on  T  and  if  their 
Wronskian  does  not  vanish  on  T,  then  the  process  {^(t)}  can  be  generated  by  a 
physically  realizable  system  characterized  by  (2-3)  with  random  initial  condi¬ 
tions  and  no  input;  i.e,,  as  the  transient  response.  The  a^(f)  in  (2-3)  can  be 
found  by  (2-31)  and  the  covariances  of  the  initial  conditions  are  given  by 

[r;;.]  =  wr[ri;.]  w  (4-6) 

where  F#  =  ElX^^iO)  A'  ^  ^  ( 0  )  ]  and  [If]  is  the  matrix  whose  i,j  element  is 

Of  course  {A'(t)}  can  always  be  generated  by  the  physically  realiz¬ 
able  system  composed  of  n-function  generators,  n-mul t ipl ier s  with  random  magni¬ 
tudes  4^,  and  a  summing  amplifier.  This  resolves  the  shaping  filter  problem 
for  this  case. 


While  it  is  not  directly  related  to  the  shaping  filter  problem,  it  is 
interesting  and  worthwhile  to  consider  the  predictability  of  processes  whose 
covariance  functions  are  of  the  form  given  in  (4-1).  In  view  of  (4-5),  it  is 
not  surprising  to  find  that  such  a  process  is  essentially  predictable  exactly. 

In  fact,  if  the  sample  functions  of  the  process  are  indeed  those  given  in  (4-5), 
then  being  able  to  predict  the  future  values  of  a  sample  function  A(t)  is  just 
a  matter  of  being  able  to  find  the  values  of  the  4^  for  the  particular  sample 
fu  net  inn  A'(t)  of  interest  in  terms  of  the  observed  values  of  A(t).  Now  if  the 
i/  j ( t  )  are  linearly  independent  on  some  interval  observation  I  C  T,  then,  as 
before,  there  exist  at  least  n-values  of  t  €  T,  say  ,  such  that  the  matrix 
is  nonsingular^  Hence,  given  the  observed  values  of  X(t)  at  the 
points  the  4,  can  be  determined  exactly  by  the  expression 


I  (  t  I  )  •  •  •  </„  (  *  |  ) 


?n(fn>  •••  ?„<*;> 


-  1 


*(<;) 


*:> 
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and  thus  the  predicted  value  Af(t)  given  by 


X(t)  =  [q,(  t )  •••  qn(t)) 

( t { )  •  •  •  9n( t  J  ) 

-1 

• 

q  (  t #  )  •  •  •  Q  It1) 

(4-8) 


equals  -Jf(t)  for  all  t  e  7\  Since  this  will  be  true  for  every  sample  function 
of  the  process,  it  follows  that  Jf(t)  =  $(t)  with  probability  1  for  all  t  €  T. 
Even  if  the  sample  functions  of  the  process  are  not  known  to  be  of  the  form 
given  in  (4-5),  as  long  as  the  covariance  function  of  the  process  is  the  same, 

A 

the  predicted  value  X ( t )  still  equals  A^(t)  with  probability  1  for  all  t  e  T* 

To  prove  this*  it  is  sufficient  to  show  that  £|jf(t)  -  ( t )  |  2  =  0.  Now 


E\X(t)  -  *(t)|2  =  r(M)  -  2 E  X(t)X(t)  +  E  X 2(t) 


(4-9) 


Computing  E  X(t)  X(t)  it  is  found  that 


A 

E  Xit)  X(t)  =  •••  <jn(  t )] 

-  1 

n*;. 

t)~ 

;> 

• 

• 

_r(C 

t)_ 

and  from  (4-1)  it  follows  that 
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ru;,  t)~ 

q,(  t ) 

a 

• 

_nc  t>_ 

=  t<ty(  t ;  n  tr  j »] 

a 

a 

jJ* ]  _ 

(4-11) 


Substituting  (4-11)  into  (4-10)  and  making  use  of  (4-1),  it  follows  that 
E  Jf(t)  ^(t)  =  Ht,  ().  Similarly,  it  is  also  easily  shown  that  E  X2(t)  ~ 

H t,  t)  and  hence  -  X{  t ) | 2  =  0  for  all  t  e  T»  Thus  it  has  been  shown 

that  if  the  qi(t)  are  linearly  independent  on  an  observation  interval  I,  then 
the  process  is  predictable  with  probability  1  for  all  t  e  T  by  a  linear  com¬ 
bination  of  n-observations  of  £(t)  on  the  interval  I.  Since  the  q^(t)  are 
linearly  independent  on  T,  this  clearly  implies  that,  if  a  process  has  a  co- 
variance  function  of  the  form  given  in  (4-1),  then  its  sample  functions  hate 
the  representation  (4-5)  where  equality  holds  with  probability  1, 

On  the  other  hand,  suppose  that  the  q4(t)  are  linearly  dependent  on  cn 
observation  interval  J.  Then  the  sample  functions  of  the  process  have  the 
representation 
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r 

X(t)  =  E  q{it)  ,  r  <  n  (4-12) 

i*  1 

for  tel  where  the  =  l,...,r)  are  linearly  independent  on  I  and  the 

B i  are  linear  combinations  of  the  Ai\  the  equality  in  (4*12)  holding  with 
probability  1.  Now,  suppose  the  process  is  predictable  with  probability  1  for 
all  t  e  T  in  terms  of  a  linear  combination  of  values  of  A'  ( t )  for  t  e  J,  Then 
the  representation  (4-12)  holds  for  all  t  e  7’.  But,  providing  [1"*^]  is 
positive- definite,  this  contradicts  the  assumption  of  linear  independence  of 
the  ( t ) ( i*l , .. . ,n )  on  T.  This  result  is  the  converse  of  that  of  the  preceding 

paragraph  for  the  case  where  [I" \j]  is  positive-definite.  Since,  if  [T1^]  is  not  positive- 
definite,  the  number  of  terms  in  its  covariance  function  can  be  reduced  to  the  point  where 
it  is  by  defining  new  <^(t)  as  linear  combinations  of  the  original  q^(t),  the  assumption 
of  nonnegati ve-defini teness  of  [r^]  entails  no  loss  of  generality* 

An  interesting  generalization  of  the  above  is  obtained  by  letting  n«°°. 

Then  is  of  the  form 

00  00 

ru ,,  tz)  r ij  q i  (t,)  9j  ( 1 2^  (4-i3) 

i*  1  jm \ 

Of  course,  in  order  for  (4-13)  to  be  meaningful,  the  mode  of  convergence  of  the 
double  series  must  be  specified.  A  natural  mode  of  convergence  is  pointwise 
on  T  x  T  and  this  shall  be  the  mode  specified.  Again,  the  q^(t)  are,  without 
1 o 8 s  of  generality,  assumed  to  be  independent  on  T.  Using  the  argument  used 
before  and  letting  n*®,  it  is  easily  shown  that  if  the  matrix  ^ ]  i,  j  *  1, 

...,  *  is  symmetric  and  nonnegati ve- definite  for  all  *,  then  r(tJ#*2)  is  a 
covariance  function.  Now  consider  the  processes  { An ( t ) }  whose  sample  functions 
are  of  the  form 


9i 


(t) 


(4-14) 


where  the  A±  are  random  variables  satisfying  E  A^  Aj  -  r^.  Now,  since  for 
all  t 

n  "  _ *  0 

£|A'n(t)-*.(t)|2  =  E  E  rij  lilt)  qj  (*>„,.-•  (4-15) 

t  1  1 

the  convergence  to  zero  in  (4-15)  being  a  consequence  of  the  convergence  of 
(4-13),  the  ^n(t)  converge  in  the  mean  to  some  limit  sample  functions  X(t)  of 
a  limit  process  {JT(t)}.  Further,  it  follows  that  the  covariance  function  of 
{X(t)}  exists  and  is  that  given  in  (4-13).  Formally  this  can  be  stated  as 


Xit) 


1 . i.m. 


-  a 


XnM) 


E 

i  ■  1 


n 


(4-16) 
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This  resolves  the  shaping  filter  problem  for  this  generalized  case. 

Turning  now  to  consideration  of  the  predictability  of  processes  whose  co- 
variance  functions  are  of  the  form  given  in  (4-13),  a  result  analogous  to  that 
given  above  for  n  <  oo  is  fairly  easily  established  providing  certain  additional 
mild  requirements  are  satisfied  on  the  observation  interval  I.  To  see  how  to 
proceed,  suppose  for  the  moment  that  the  sample  functions  of  the  process  are  of 
the  form  given  in  (4-16)*  Then  formally,  if  there  existed  a  set  of  functions 
flit)  such  that  fj  filt)  dt  =  t*ie  cou^  be  found  by  multiplying 

both  sides  of  (4-16)  by  the  /^(t)  respectively  and  integrating  over  the  obser¬ 
vation  interval  J.  Once  the  4^  have  been  determined,  the  predictability  follows 
immediately.  Now  the  additional  requirements  mentioned  above  are  just  those 
needed  to  rigorously  justify  this  procedure  in  general  [that  is,  even  when  it  is 
not  known  a  priori  that  the  sample  functions  of  the  process  are  of  the  form  given 

in  (4-16)].  This  motivation  leads  formally  to  the  consideration  of 

n 

Xn{t)  =  Z  qAt)  fj  fAt )  Xit)  dt  (4-17) 

i  *  1  J 


as  a  predicted  value  of  A'(t)  where  hopefully  the  error  of  prediction  goes  to 
zero  as  n  -+  oo  • 


Specifically,  assume  that  the  g^(t)  are  linearly  independent  on  I  in  the 
sense  that  for  all  finite  n  there  exists  no  set  of  constants  not  all  zero 
n 

such  that  X  di  q^(t)  =  0  almost  everywhere  on  I*  and  that  J j  q*(t)dt  <  oo 
i  *  1 

for  all  i.  Then  as  shown  in  Appendix  1  there  exists  a  set  of  functions  /^(t) 
defined  on  I  such  that  fj  f\it)  dt  <  oo  for  all  i  and  Jj  /^(t)  qj(t)  dt  =  S^, 
Finally,  assume  that  one  of  the  following  statements  is  true:  Cither  (i) 
n  ® 

|Z  2  r.  .  q  i  ( 1 1  )  q  •  1 1  2  )  |  £  5  j  (  1 1  )  almost  everywhere  on  1  for  all  n  and  all 
i  -  i ;  3  i  J  J 


ft 

t2el  and  |  2  f  S2(t^)  almost  everywhere  on  I  for  all  n,  j  where 


i*  1 


S.(t|)  and  S2(t2)  are  integrable  on  1,  or  (ii)  dt  ,  <  oo  and  the 


n  ® 


series  2  £  T,  q A  t 2)  converges  weakly  to  P( t } ,  1 2 )  on  /  for  all  t 2tl 

nn 

and  J  /x/P2^  t  2,  t ,  )  dty  dt  2  <  oo  and  the  series  I  I  -  g  €  (  r  f  )  gjt2)  converges 
weakly  to  on  /  x  /.*•  Under  these  assumptions  consider  the  expression 

£|JKi)  -  in(  r )  |  2  =  Vit.t)  -  2EXn[t)  Xit)  *  E  X^it)  (4-18) 


Not®  th®t  thi®  definition  of  linear  independence  differs  slightly  from  the  clsssicsl  defini¬ 
tion  used  above  for  the  case  where  n  <  ®  , 

*#Note  that  convergence  in  the  wean  implies  weak  convergence, [l8 ,  p.l75]. 
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A 

Computing  E  ^n(t) 

A 

E  Xn(t  )  X(t  )  = 


there  results 
n 
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( t )  fj  f i (r)r(r,  t  )  dr 


=  Z  9  *  < « >  E  E 

n  ® 

=  E  E  r\  ,4u) 

i- l  y«l  J 


rK  9j(t>  /l  /ilT>  HiT)  dT 

<J, <*>”  tnt.t) 

J  n  -*  oo 


(4-19) 


the  interchange  of  order  of  summation  and  integration  being  justified  by  the 

assumption  of  bounded  convergence  or  weak  convergence  of  the  relevant  series. 

A 

Similarly,  computing  E  there  results 
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l  i 

i  *  1  i  ■ 

l  l 


9*1*)  9j(t)  /l  dTl/r  dTz  /*(t1  1  /j(T2)r(Ti,T2> 

00  00 

qj(t)  fj  dr ,  E  E  1%  -  /4(r,  )  9fc(T,) 


fl*T2fj{T  2>  ^(T2) 

n  n 

=  E  E  r{  ,  <j.(t)  q  At)  *  r(t.t)  (4-20) 

*•1  j-i  1  :  n  -  a> 


Hence, 


E\X(  t) 


an  *  6  t 
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which  proves  that  under  the  above  mild  assumptions,  processes  whose  covariance 
functions  are  of  the  form  given  in  (4-13)  are  predictable  in  the  mean.  As  before, 
linear  independence  of  the  <?{(t)  is  of  crucial  importance;  it  being  used  in  the 
proof  of  the  existence  of  f^(t)  which  satisfy  Sj  /$  (  t )  qj(t)  dt  =  8^. 


The  above  result  includes  as  a  special  case,  two  classes  of  processes  which 
ure  known  to  be  predictable  in  the  mean  for  any  nondegenerate  observation  inter¬ 
val  1C  T;  namely,  those  with  analytic  covariance  functions  and  those  stationary 
processes  whose  spectral  distribution  functions  are  step  functions.  Finally,  it 
is  interesting  to  note  that  if  one  has  expanded  what  Wiener  calls  an  innovation 
process;  e.g.,  a  stationary  process  with  absolutely  continuous  spectral  distribu¬ 
tion  function,  in  an  infinite  series  over  an  interval  T  by  the  Karhunen-Loeve 
expansion  theorem  or  by  any  other  method  leading  to  reasonably  convergent  series 
for  ntjjtg),  then  while  the  q^it)  in  the  expansion  are  linearly  independent  on 
T,  they  cannot  be  linearly  independent  on  any  proper  subinterval  of  T.  If  they 
were,  the  process  would  be  predictable  in  the  mean  in  terms  of  values  on  the  sub¬ 
interval  which  contradicts  the  assumption  that  the  process  is  an  innovation  pro¬ 


cess. 


4. 3  THE  “ALMOST  STATIONARY”  CASE 


Another  case  for  which  the  shaping  filter  problem  can  be  resolved  by 
elementary  methods  is  that  where  r(t(,t2)  is  of  the  form 


H  t 


.  I  a,."*1'.-*1.  J  J  c4,  .-J*- 

i  ■  1  ( ■  I  j  ■  t 


(4-22 ) 


where  >  0  and  the  and  D ^  are  real  or  occur  in  complex  conjugate  pairs. 
Here,  assuming  for  the  moment  that  r(tJ#t2)  is  a  covariance  function,  the  non¬ 
stationary  character  of  r(tj,t2)  arises  solely  due  to  tronsients  which  die  out 
as  t,,t2  -*  oo  ,  the  corresponding  process  being  asymptotically  stationary. 


Now  a  sufficient  condition  that  r(t,,t2)  be  a  covariance  function  is  that 

n  -  a  .  I  r  I 

the  Fourier  Transform  of  2  e  1  be  everywhere  positive  and  the  matrix 

i  ■  t 

[C{j]  be  nonnegative-definite.  In  this  case  one  can  find,  by  the  usual  method 

J  -a . | v | 

of  factoring  the  Fourier  Transform  of  z  Di  e  1  »  a  weighting  function  for 

i  ■  1 

a  shaping  filter  of  the  form 

/  n 

£  d  e-*^ 

If ( r )  =  <  -  (4-23) 

0 


,  T  >  0 
,  T  <  0 


whose  corresponding  inverse  is  stable.  If  a  white  noise  process  is  applied  to 
the  shaping  filter  at  t  =  0  with  the  shaping  filter  assumed  to  be  at  rest  at 
t  =  0,  then  the  covariance  function  ( 1 1  #  1 2 )  of  the  output  process  is 

r  ( t  t  )  =  Z  Di  -  fj  t  di,-^  ."**  *'"*'  *2  (4-24) 

1  '  c  j.  ,  *  j  .  .  j-.CLj+CLj 
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Note  that  the  function  £  £ 

i*l j«i 


n  n  d{  di  -a.  t.  -a,  t 

e  e  J  is  a  covariance  function 


ai  + 


on  T  x  T.  This  follows  from  the  fact  that  if  a  white  noise  process  is  applied 
to  the  shaping  filter  over  the  remote  past  and  then  removed  at  t  -  0,  then  the 
covariance  function  of  the  resulting  transient  output  process  for  t  >  0  will  be 


n  n  d  di 

1  1  —J— L 


■ij-i  ai  +  -j 


a , 


ai  *i  e  j  *2  #  Subtracting  (4-24)  from  (4-22)  there  results 


r,  ( t , , t , ) 


n  n 

Z  Z  r 

i  -  I  j m  I  J 
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which,  by  the  assumptions  made  on  [C^]  »  is  a  covariance  function.  Making  use 
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of  the  results  of  Section  4.2,  this  establishes  that  with  s  white  noise  source 
and  s  set  of  random  initial  conditions  which  are  uncorrelated  with  the  white 
noise  source,  a  process  whose  covariance  function  is  of  the  form  given  in  (4-22), 
can  be  generated  by  the  system  whose  weighting  function  is  that  given  in  (4-23). 
Of  course,  even  if  the  matrix  [C^]  is  not  nonnegative-definite,  the  result 
still  holds  as  long  as  [r^j  is  nonnegative-definite* 

4.4  THE  “NONDEQENEKATE”  CASE 

The  final  case  where  the  shaping  filter  problem  can  be  resolved  by 
elementary  methods  is  baaed  on  consideration  of  (3-3).  Suppose  that  one  were 
given  a  function  expressed  in  the  form  given  in  (3-4)  where  each  of 

the  Pj(t)  is  a  sum  of  at  least  n-terms.  Suppose  further  that  upon  division  of 
P{(t)  by  Qjit)  andpj(t)  by  q^(t),  it  happens  that  the  resulting  sums  have  a  term  in 

common.  Then  by  (3-3)  it  might  be  assumed  that  this  term  is  f  dt(i{  ( t  )/3  (  t ) 

+  r ijm  If  this  happens  for  all  j  ^  i,  then  the  one  term  of  p{(t)  which,  after 
the  appropriate  division,  did  not  appear  to  be  a  term  which  p^(t)  had  in  common 

with  some  Pj(t )  might  be  assumed  to  be  <**(»)[  /*  /S|(r)  dr  +  r<4].  If  it  were, 

then  0{(t)  could  be  found  by  dividing  it  by  q^t),  differentiating  with  respect 
to  t,  and  taking  the  square  root  of  the  resulting  derivative.  Once  the  fi^t) 
have  been  determined,  the  determination  of  the  i.  obviously  trivisl. 

Whenever  the  procedure  sketched  above  works,  it  will  be  ssid  thst  one  is 
dealing  with  a  nondegenerate  esse.  At  first  glance  it  might  appear  that  this 
is  indeed  a  special  case  since,  for  example,  stationary  processes  and  almost 
stationary  processes  are  clearly  degenerate  cases.  However,  there  appears  to 
be  a  reasonably  large  class  of  jjonst_ationsr^  processes  which  lead  to  nonde¬ 
generate  cases.  For  example,  consider  the  case  where  is 
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+  +**[*•"*  «'»'•]  ;  t,  >  t,  >  0 
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I  ♦  «!/7l  ♦  *|l**/7  +  >  0 


Here 


t  ,  p  ,  ( » )  =  ♦  t%n 

t 2  .  pt(t)  =  tt/7  *  *j''* 


(4-27) 
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Upon  dividing  pt(t)  by  t2  and  p2(t)  by  t,  it  is  observed  that  t7/7  is  a  common 

t 

term.  Hence,  t6/5  appears  to  be  q  1  ( t )  [/  dr/32(r)  +  ru].  Proceeding  under 

o 

this  assumption,  it  is  found  that  /3,  (  t )  =  ±  t2  and  PM  =  0.  Similarly,  it  ia 

found  that  yS2(  t)  =  ±  t4  and  T22  =  0.  Letting  t9/7  =  q2(t)[/  drfi^(r) /3z(r)]  +  T12, 

o 

it  is  found  that  this  relationship  is  indeed  satisfied  if  T, 2  =  0  and  similarly 
for  t9/7.  Hence,  the  weighting  function  of  an  appropriate  shaping  filter  is 

0(  t,r)  =  trz  +  t2r 4  ,  t  >  r  >  0 

=  0  ,  t  <  r  (4-28) 

There  does  not  appear  to  be  any  simple  criterion  for  determining  whether 
a  given  function  r(t},t2)  is  the  covariance  function  for  a  nondegenerate  case 
short  of  attempting  to  carry  out  the  above  procedure.  Since  the  procedure  is 
rather  simple  and  direct,  it  is  reasonable  to  just  proceed  as  if  it  were  a 
nondegenerate  case  and,  if  it  fails  at  some  step,  then  one  concludes  that  he 
is  dealing  with  a  degenerate  case  and  a  more  complex  procedure  is  required.  If 
it  works,  the  shaping  filter  problem  has  been  resolved  rather  easily.  As  the 
example  shows,  sometimes  nonstationary  cases  are  easier  than  stationary  cases. 

Note  that  when  it  works,  the  above  procedure  always  yields  a  physically 
re al izable  shaping  filter.  Also  note  that  after  the  have  been  determined, 
it  is  necessary  to  check  and  make  sure  that  the  matrix  is  nonnegative- 

definite  and,  hence,  represents  the  covariance  matrix  of  a  set  of  random  initial 
conditions  on  the  shaping  filter. 


CHAPTER  5 


SOME  FUNDAMENTAL  RESULTS  ON 
EXISTENCE  AND  UNIQUENESS 


5. I  INTRODUCTION 

As  is  well  known,  physically  realizable  shaping  filters  do  not,  in  general, 
exist  for  processes  with  arbitrary  covariance  functions.  For  example,  as  pointed 
out  in  Section  1.2,1  for  the  stationary  case,  physical  realizability  of  the  shap¬ 
ing  filter  requires  that  the  spectra]  distribution  function  be  absolutely  con¬ 
tinuous  and  satisfy  the  Paley-Wiener  criterion  given  in  (1-13),  Apparently  no 
simple  criterion  analogous  to  that  of  Paley  and  Wiener  has  been  developed  for 
the  general  nonstationary  case.  This  is  not  too  surprising  considering  the 
difficulty  of  the  problem.  In  this  chapter,  the  question  of  the  existence  of 
physically  realizable  shaping  filters  is  investigated  for  the  class  of  separ¬ 
able  covariance  functions,  and  for  this  class,  it  is  shown  that,  providing  one 
remarkably  simple  requirement  is  met,  a  physically  realizable  shaping  filter 
does  indeed  exist.  In  addition,  the  question  of  uniqueness  of  the  shaping 
filter  is  also  discussed.  The  restriction  to  the  class  of  separable  covariance 
functions  certainly  seems  reasonable  in  view  of  the  fact  that  in  this  case  thi 
resulting  shaping  filter  is  usually  rather  easily  realized  physically.  This  is, 
of  course,  of  extreme  importance  in  engineering  applications. 

It  is  rather  interesting  and  enlightening  to  examine  the  treatment  of  this 
question  for  the  nonstationary  case  by  the  authors  whose  work  is  summarized  in 
Chapter  1,  Darlington  was  apparently  well  aware  of  the  problem  and  did  provide 
answers  for  two  rather  restrictive  cases.  They  were  restrictive  in  the  sense 
that  he  assumed  physical  realizability  of  the  underlying  signal  and  noise-shaping 
filters  and  either  periodicity  or  regularity  at  oo  of  the  corresponding  differential 
equation.  His  answers  clearly  leave  much  to  be  desired,  Batkov  simply  avoided 
the  problem  by  making  implicit  the  assumption  that  the  covariance  function  was  of 
the  required  form  (a  trick  quite  commonly  used  in  writing  technical  papers).  One 
wonders  if  he  was  even  aware  of  the  problem,  Leonov  wasn't  concerned  about 
physical  realizability  of  the  shaping  filter  (and  didn't  discuss  or  obtain  it) 
because  it  wasn't  required  for  his  application.  Actually,  for  his  application, 
he  didn't  need  shaping  filters  at  all. 

Finally,  it  should  be  noted  that  physical  realizability  of  the  shaping 
filter  is  automatically  achieved  for  the  special  cases  discussed  in  Chapter  4, 
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5.2  CASTING  AND  RECASTING  THE  PROBLEM 


Neglecting  initial  conditions  for  the  moment,  establishing  the  existence 
of  a  physically  realizable  shaping  filter  amounts  to  establishing  the  existence 
of  a  solution  of  the  nonlinear  Volterra  integral  equation  of  the  first  kind 

^2 

rtt,,  t2)  =  f  dr  ITUj.t)  ITU2,t)  dr  ,  t,  >  t2  >  0  (5-1) 

0 

When  the  covariance  function  is  separable;  i.e#> 


n  1 1,  t2) 


r» 

.  I 


<7<  (  <t  )  p<(  t2)  ;  t ,  >  t2>  0 


(5-2) 


it  is  reasonable  in  view  of  Chapters  2  and  3  to  consider  solutions  of  the  form 


ri 

f  ^  q*(  t  )fii{r)  ;  t  >  r  > 


ITU.t)  =< 


;  t  <  t 


(5-3) 


In  this  case,  the  integral  equation  becomes 
n  n  n 


^  q  i  (  *  I  )  p^lt,)  =  ^  g  At,)  ^  g^U.)  /  dr  /2At)  /3At)  (5-4) 

i-i  <-i  J  0  J 


and  upon  making  use  of  the  linear  independence  of  the  and  adding  on 

initial  conditions  terms,  there  results 


P{{  t)  =  Z  q  jit) 

j«i 


r  t 


/  dT/3i(T)/3j{r )  + 


;  t  >  0.  i  =  1, 


(5-5) 


Thus  for  the  case  of  separable  covariance  functions,  the  problem  has  been  re¬ 
duced  to  establishing  the  existence  of  a  solution  of  the  simultaneous  nonlinear 
Volterra  integral  equations  of  the  first  kind  given  in  (5*5).  This  certainly 
represents  a  reduction  over  (5-1)  since  (5-1)  actually  represents  an  infinite 
set  of  simultaneous  integral  equations,  one  for  each  value  of  t|# 

The  usual  procedure  in  the  study  of  Volterra  integral  equations  of  the 
first  kind  is  to  first  convert  the  integral  equation  into  an  integral  equation 
of  the  second  kind  and  then  apply  the  standard  techniquea  known  for  Volterra 
integral  equations  of  the  second  kind*  For  linear  equations,  this  conversion 


is  easily  carried  out,  [ 2 5 • p •  1 6] •  That  such  a  procedure  can  also  be  carried 
out  for  the  equations  in  (5-5)  is  perhaps  not  obvious,  but  nevertheless  it 
can  be  accomplished  as  follows. 


Upon  differentiating  the  equations  in  (5-5),  there  results 


n 

pj'Mt)  =  £  qi'Mt) 

i-i 


i 

f  dr/ 3i(T)/3j{T)  +  T 


+  /6{(t)  Z  qj(t)/3j(t) 


(5-6) 


Now  examination  of  (5-6)  shows  that  the  multiplier  of  >0^  ( t )  is  the  same  for 
n  * 

all  i;  namely,  2  q  At)/3A  t)  m  Let  k(t)  =  2  qAt)fiAt)*  The  only  problem 

jm\  J  J  jm \  J  J 

is  that  k(t)  is  unknown*  If  k(t)  were  known  and  nonxero  for  all  t  >  0,  then 
the  desired  conversion  to  integral  equations  of  the  second  kind  would  be 
complete  upon  division  by  k(t).  While  at  first  glance  it  may  appear  that, 
since  k(t)  involves  the  unknown  /3j(t),  there  is  no  hope  of  being  able  to 
determine  it,  it  can  be  determined.  Solving  (5-6)  for  /3^(t)  it  is  found  that 


I 

Substituting  back  into  (5-6)  there  results 
"  t 

pj”  U)  -  £  <?}n(t)  /  +  T, 


J  dr/3i(T)/3j{T)  +  rti  |L  (5-7) 


j-' 


=  k-*lt)  /p|n(t)  -  Z  qj'Ut) 


a 

f  drfii  +  ft  j 


PJ 


S* »( 


t)  -  Z  q}(t)  <ji"(t) 


*-1 


f  drfi^T)(ik(r)  +  , 


'(5-8) 


which,  upon  cancelling  and  rearranging,  yields 

n  n  n 

k2(t)  =  Z  qAt)  pi'Mt)  -  Z  ql'Ut)  £  qAt)  X 

j-i  3  1  J»-1  J-»  1 

f  drpj(T)/3k[T)  +  rjh  j 

But  the  second  term  in  (5-9)  is  just  p,(t)  and  hence 


(5-9) 
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kit)  =  ±  /  Z  pj'Mt)  “  qj'Mt)  PjU)J 


(5-10) 


Thus  k(t)  can  be  determined  from  r( t j , 1 2 ) •  Making  use  of  (3-4)  and  ( 3 - 6 ) , 
k(t)  can  be  expressed  in  the  form 


kit)  =  ±  ^r<°.'»(t<T)|Tt  ~  -  r(o..)u#T)|Ti  t  =  ±  yjoj(t,  (5-11) 


It  is  quite  important  to  note  that  if  the  yS^(t)  are  to  be  real  then  (5-7) 
together  with  (5-11)  requires  that  J  (t)  >0  for  all  t  >  0.  This  completes 
the  conversion  of  the  integral  equations  of  the  first  kind  given  in  (5-5)  to 
integral  equations  of  the  second  kind  as  given  in  (5-7). 


Of  course*  there  still  remains  the  problem  of  what  to  do  in  case  kit)  *  0* 
In  this  case*  the  equations  given  in  (5-6)  reduce  to  a  set  of  integral  equa¬ 
tions  of  the  first  kind  of  the  form  given  in  (5-5)  with  q^(t)  and  p^(t)  replaced 
by  qly)(t)  and  pj1J(t)  respectively.  Hence*  the  logical  thing  to  do  is  to 
re-apply  the  conversion  procedure  used  in  the  previous  paragraph  on  (5-5). 

When  this  is  done  one  obtains 


Piit) 


where 


t) 


<^pj2,(t) 


l 

i-1 


<}j2>(t) 


t 

j  drpiiT)/3ji r) 

o 


(5-12) 


k  ,(t) 


,<"(<)  p|2,(t)  -  q]  2  *  (  t )  p<"(t) 


(5-13) 

For  the  same  reason  as  before*  it  is  required  that  Jf  2(t)  >  0  for  all  t  >  0. 
Naturally*  if  k  ( t )  ^  0,  then  one  re-applies  the  procedure  to  the  new  equa¬ 
tions  etc. 


When  k(t)  -  0  [or  k^it)  =  0,  etc.]  for  some  values  of  t  but  not  identically* 
then  one  is  dealing  with  a  more  complicated  type  of  integral  equation  which 
Picard  called  an  equation  of  the  third  kind.  Such  cases  have  been  studied  for 
linear  equations  by  Lalesco  [26]. 

The  problem  now  has  been  reduced  to  establishing  the  existence  of  a  solu¬ 
tion  of  the  integral  equations  of  the  second  kind  given  in  (5-7).  To  do  this*  use 


will  be  made  of  some  results  due  to  T.  Sato  [27].  Since  Sato  makes  use  of 
Schauder’s  fixed  point  theorem  in  his  treatment  of  existence  questions,  a  short 
discussion  of  fixed  point  theorems  is  in  order. 

ft.  3  DISCUSSION  OF  FIXED  POINT  THEOREMS 

The  basic  idea  underlying  fixed  point  theorems  can  be  nicely  demonstrated 
by  the  following  simple  example  [28,  p  •  1 1 6 ] •  Let  C  be  the  set  { X  :  0  <  X  <  1} 
and  let  cr[X)  be  a  continuous,  single-valued  transformation  of  C  into  itself  (i.e., 
cr{x)  is  a  continuous,  singl e- valued  function  defined  on  [0,1]  for  which  cr(x)e[0,l] 
for  all  xc[0,l]).  Then  there  exists  an  x QeC  such  that  xQ  =  cr(xQ).  xQ  is  called 
a  fixed  point  for  the  transformation  cr(x).  The  truth  of  this  result  is  obvious 
from  Figure  2. 


It  is  also  obvious  that  xQ  may  be  either  1  or  0  and  that  it  is  not  necessarily 
unique  (there  are  four  fixed  points  in  Figure  2). 

The  generalization  of  this  simple  result  to  more  general  sets  C  in  more 
general  underlying  topological  spaces  has  led  to  the  development  of  rather  power¬ 
ful  (fixed  point)  theorems  for  establishing  the  existence  of  solutions  (fixed 
points)  of  functional  equations  in  general  and  integral  equations  in  particular. 
For  integral  equations  C  becomes  a  class  of  functions  and  a  is  an  integral 
operator;  e.g.  ,  the  right-hand  side  of  (5-7);  and  asserting  the  existence  of  a 
fixed  point  for  cr  is  clearly  equivalent  to  asserting  the  existence  of  a  solution 
of  the  corresponding  integral  equation.  One  of  the  most  general  fixed  point 
theorems  and  the  one  apparently  used  by  Sato  was  proven  by  Schauder  and  can  be 
stated  as  follows:  [29,  p.260). 

Scfcaader* s  Theorea:  Let  C  be  a  nonempty,  compact,  convex  set  from 
a  locally  convex  apace  X  and  let  a  be  a  continuous,  single-valued 
transformation  of  C  into  C .  Then  there  exists  an  xQ€C  such  that 
<M*0)  =  *o* 
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In  applying  Schauder’s  Theorem,  the  essential  problem  is,  of  course,  to  find  an 
appropriate  class  C  for  the  problem  at  hand* 

It  is  interesting  to  note  that  the  requirements  of  compactness  and  convexity 
of  C  stated  in  the  theorem  could  have  been  anticipated  on  the  basis  of  the  simple 
example  given  above* 

5.4  BESOLUTION  OF  THE  EXISTENCE  PROBLEM 

Because  Sato's  treatment  of  the  existence  of  solutions  of  the  integral  equa¬ 
tions  he  studied  is  rather  sketchy  in  nature,  a  development  of  it  is  given  below* 
The  development  is  essentially  that  given  by  Sato  except  that  many  of  the  obvious 
(to  Sato!)  steps  are  filled  in,  an  error  is  corrected,  and  the  appropriate  space  X 
and  set  C  are  clearly  defined*  The  theorem  resulting  from  this  development  is  then 
applied  to  the  set  of  integral  equations  given  in  (5-7)  and  the  existence  of 
physically  realizable  shaping  filters  is  thereby  deduced* 

The  following  notation  will  be  useful 

Ir :  the  closed  interval  0  <  x  £  r 

Af:  the  closed  domain  0  <  t  <  x  <  r  in  the  plane  (x,t) 

D:  the  closed  domain  in  the  space  (x,  t,  u  ,  un) 

defined  by  (x, t )€  Ar#  |  -  /^(x ) |<  p  where  the 

/{(x)  are  continuous  functions  on  Ir  and  p/2  >  “•* 

[■“  /<(*)  -  ■*“  fiix))  >  0 

Jr 

(For  any  p  >  0  there  is  obviously  an  Ir ,  r  >  0 
such  that  the  latter  inequality  is  satisfied*) 

Now  consider  the  set  of  integral  equations 

x 

U{(x)  =  f^x)  +  J  Ki  [x,  t  ,i*|  ( t  )#  Un(  t)]dt  (5-14) 

o 

where  f(^[x,t#u,#  ••*,  un]  is  continuous  on  D*  Hence,  there  exists  an  M  such  that 

\Ki[x,  t,ux,  •.*,*,  un]  |  <11  for  all  i  and  all  (x,  t,ut#  un)eD.  Let  u(x)  be 

the  vector-valued  function  whose  components  are  Uj(x)*  Having  made  the  above 
aasumptions,  the  problem  now  is  to  find  a  space  X  and  a  set  C  which  satisfy  the 
hypotheses  of  Schauder’s  Theorem* 
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Let  X  be  the  space  of  all  continuous  vector  functions  u(x)  on  Ir  and  let 
X  have  the  topology  of  uniform  convergence  on  compact  (i.e.,  on  all  compact 
subsets  of  I r)  [30,  p •  2  2 6 ] •  Then  X  is  clearly  a  locally  convex  space.  (For  a 
definition  of  a  locally  convex  space  see  [29,  p.257].)  Now,  let  F  be  any  set 
of  vector  functions  u [x)eX  which  satisfy  the  conditions 

!«**(*)  -  /i  t  * )  I  £  p/2  ;  ut(0)  =  /t  (0)  (5-15) 

on  lr,  where  r'  =  min  (r,  p/2M )*  Then  the  right-hand  side  of  (5-14),  considered 
as  a  transformation  cr,  obviously  transforms  F  into  a  set  F  of  vector  functions 
U(x)eX  which  also  satisfy  (5-15)  on  Ir,.  Furthermore,  the  set  F  is  seen  to  be 
equicont inuous  on  Ir,  (in  fact,  F  is  uniformly  equicont inuous  on  I r) •  Hence, 
for  every  e  >  0,  there  exists  a  8(6)  >  0  such  that  for  all  u(x)eF9  | ( x 4 )  - 
U|(x2)|  —  e  f°r  everY  x\»  xz satisfying  |x,  -  *2|  £  8(6).  Let  C  be  the  set 
of  all  functions  u{x)eX  which  satisfy  (5-15)  and  are  such  that  lu^Xj)  _ 

<  e  for  every  x},  x2  e  satisfying  | x  1  -  x  z\  <  8(6).  Then  cr  clearly  trans¬ 

forms  C  into  itself.  Furthermore,  C,  being  equicont inuous ,  is  compact  by  Ascoli’s 
Theorem  [30,  p.234]  and  is  easily  seen  to  be  convex.  Finally,  since  the 
K^[x,t,u)t  un]  are  continuous  on  D  and  D  is  closed,  they  are  uniformly 

continuous  on  D.  Hence,  cr  is  a  continuous  transformation  of  C  into  itself  and 
is  clearly  aingl e- val ued.  Thus,  by  applying  Schauder's  Theorem  the  following 
result  is  deduced. 

Theores  1:  Let  f ^(x)  be  continuous  on  Ir  and  let  K^[x9t,Uy,  ...,un]  on  D.  Then 
on  Irj  there  exists  at  least  one  continuous  solution  of  the  integral  equations 
given  in  (5-14)  where  r*  =  min  (r,  p/2M). 

Of  course,  the  solution  of  (5-14)  can  be  extended  to  Ir  or  to  the  boundary 
of  D  by  the  standard  argument. 

By  a  straightforward  application  of  the  above  theorem  to  the  set  of  inte¬ 
gral  equations  given  in  (5-7),  the  following  important  theorem  is  easily 
deduced . 

Theorem  2:  Suppose  that  I "(  t  1 , t 2  )  is  of  the  form  given  by  (3-4),  that  q j 1  * ^ t  ) 
and  />  [ 1  )  (  f)  exist  and  are  continuous  on  [0,7’],  that  ,(t)  0  on  [0,7],  and 

n 

that  there  exists  a  nonnegative-definite  matrix  [I’-.]  such  that  />  ^  ( 0 )  - 

J  1 

F.  q  •  ( 0 )  0  for  all  i.  Then  P( t } , t 2 )  is  a  covariance  function  on  [0,7]  aid 

there  exists  a  physically  realizable  shaping  filter  on  [0,7]  whose  weight!  ig 
function  is  ot  t  lie  form  given  in  (2-19)  where  the  p^(t)  are  continuous  on 
[0,7]. 
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The  only  possible  difficulty  with  the  application  of  Theorem  1  to  Theorem  2 
is  that  it  may  be  impossible  to  extend  the  solution  to  [0,7]  without  leaving  D» 
Since  D  itself  [for  (5-7)]  can  be  increased  in  the  coordinate  directions  and 
still  meet  the  requirements  of  Theorem  1  indefinitely,  this  implies  that  either  the 
solution  can  be  extended  to  [0,7]  or  else  the  fi4(t)  become  unbounded  and!  hence, 
discontinuous.  In  the  statement  of  Theorem  2  and  in  those  to  follow,  it  has  been 
assumed  that  it  is  possible  to  extend  the  solution  to  [0,7].  In  any  case.  Theorem 
2  holds  on  [0,7#]  where  p/2M  <  T1  <  T .  Cases  where  the  /3^(t)  are  unbounded  are 
not  of  great  importance  in  engineering  applications  and,  furthermore,  computational 
problems  arise  in  these  cases  anyway. 

When  J0  j(t)  1  0  on  [0 ,7]  but  J|#2(t)  >  0  on  [0,7],  then  the  following 
modified  form  of  Theorem  2  holds. 


Theorea  3:  Suppose  that  r(f1#t2)  is  of  the  form  given  by  (3-4),  that  q|2,(t) 

and  pj2,( t)  exist  and  are  continuous  on  [0,7],  that  a(t)  >  0  on  [0,7],  and 

n 


that  there  exists  a  nonnegative-definite  matrix  [r^] 

qAO)  =  0  and  pjn(0)  -  2  ri4qVH0)  =  0  for  all  i. 

J  i*l  3  3 


such  that  pj( 0)  -  2 

i  *  I  J 

Then  H  1 1 , t %  )  is  a  co- 


variance  function  on  [0,7]  and  there  exists  a  physically  realisable  shaping 
filter  on  [0,7]  whose  weighting  function  is  of  the  form  given  in  (2-19),  the 
P4(t)  are  continuous  on  [0,7],  and  a  0  for  te[0,7]. 


The  further  modification  of  Theorem  2  when  Jq  j  ( t )  =  Jj  2(t)  ■  0  on 
[0,7]  but  J2>3(f)  >  0  on  [0,7]  is  obvious.  The  case  there  J0,(t)  =  0  for 
some  te[0,7]  but  not  identically  is  not  discussed,  but  satisfactory  results 
could  possibly  be  obtained  by  following  up  Lalesco's  work  [26]. 


Finally,  suppose  that  in  addition  to  satisfying  the  hypotheses  of 
Theorem  2,  the  q i ( t )  and  p ^ ( t )  have  n- continuous  derivatives  on  [0,7]  and 
the  Wronskian  of  the  q±(t)  doesn’t  vanish  on  [0,7].  Then  by  successive  dif¬ 
ferentiation  of  Equations  (5-7)  it  follows  that  the  ( t)  have  n  -  1  con¬ 

tinuous  derivatives  on  [0,7].  Hence,  by  the  results  in  Section  2.4,  it 
follows  that  the  shaping  filter  can  be  characterized  by  a  differential  equa¬ 
tion  of  the  form  given  in  (2-1)  where  the  a^(f)  and  6^(0  are  continuous  on 

[0,7]. 

5.5  THE  QUESTION  OF  UNIQUENESS  OF  THE  SHAPING  FILTER 

Having  resolved  the  question  of  the  existence  of  a  physically  realizable 
shaping  filter,  the  question  of  the  uniquenesa  of  the  shaping  filter  naturally 
arises.  Examining  (5*5),  it  is  clear  that  there  is  no  unique  solution  because 


if  H'(t,r)  is  a  solution,  then  -I Ht,r)  is  also  a  solution.  Note  that  if  is 

the  solution  associated  with  the  plus  sign  in  (5-10),  then  -W(t,r)  is  the  solu¬ 
tion  associated  with  the  minus  sign.  However,  the  question  still  remains  as  to 
whether  the  solution  is  unique,  say,  to  within  a  multiplicative  factor.  The 
answer  is  again  no  because  there  may  be  more  than  one  nonnegative-definite  matrix 
[r{j]  which  meets  the  requirements  of  Theorem  2  and  which  lead  to  different 
solutions.  As  an  example,  consider  the  covariance  function 

ru)4t2)  =  4/3  e’U'  "  -  5/12  ’  ‘2^  t,,  t2  >  0 


The  two  matrices 


Fiji 


2  -  2/3 

-  2/3  1/4 


8  -  20/3 

-20/3  25/4 J 


'o,.*')  "  1 


Both  meet  the  requirements  of  Theorem  2.  Furthermore,  and  as  direct  substitu¬ 
tion  into  (5-7)  shows,  =  2e  *  and  /S2(t)  =  -  e2t  is  a  solution  on  [0,a>]  for 

[r^]  while  =  -  4e*  and  /32  ( t )  =  5e2{  is  a  solution  on  [0,oo]  for  [rj^]. 

Hence,  IT ,  ( t ,  r )  =  2e_U"T)  -  e-zit~r)  and  I f2(t,r)  =  -  4e'U"T)  +  5e‘2(  t-T'are 
respectively  the  weighting  functions  of  the  physically  realizable  shaping  filters 
for  these  matrices.  Taking  the  Laplace  transforms  of  If t  (  t  -  r)  and  I f2(t  -  T) 

•S  +  3  5  -  3 

there  results  Gt(S)  =  - - — - -  and  GJS)  =  - - — - - —  •  It  is  interesting 

1  IS  ♦  1)15  t  2)  2  (5  +  IMS  +  2) 

to  note  that  the  transfer  function  of  the  system  associated  with  [rf^]  has 
its  zero  in  the  left-half  plane  while  that  associated  with  [rj has  its  zero 
in  the  right-half  plane.  In  light  of  this  example,  the  question  now  arises 
as  to  whether  the  solution  is  unique  if,  say,  the  plus  sign  in  (5-10)  is  used 
and  a  matrix  is  specified  which  meets  the  requirements  of  Theorem  2.  The 

answer  this  time  is  yes  as  the  theorems  proven  below  show. 


Suppose  that  in  addition  to  satisfying  the  hypotheses  of  Theorem  1,  the 
K^[x^,t,u un]/Buj  are  continuous  on  D  for  all  i , j.  Then,  since  D  is 
closed,  they  are  bounded  on  D.  By  the  mean  value  theorem,  this  implies  that 
a  Lipschitz  condition  is  satisfied  on  D;  i.e.,  that  there  exists  an  such 

|  |K[x,  t,ut,  UB]  -  K[x,t,vt,  %]  I  I  <  I  |u  -  i/|  I  where  |  |u  -  v  |  |  = 

n 

£  |ui  -  etc.  Under  this  stronger  assumption,  the  existence  of  a  solution 

i  *  1 

to  (5-14)  on  Ir,  can  be  established  by  successive  approximations  as  follows. 

Let  u°(x)  =  fix)  and  u^’li) 
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x 

u{*'lx)  =  /,(*)  +  f  K{[x,t,u^t),  ....  uJlt )]  dt  (5-16) 

Then  J*'(x)eD  and  is  continuous  for  e  lf  ,  as  is  easily  established  by  induction* 
Furthermore 

x 

llu^'U)  -  «^(*)||  <  #,  /  ||u-*U)  -  ui-,(*)||dt  (5-17) 


|  I u 1 ( X )  -  u°(*)||  <  f  II K[x,  t,ft[  t),  ....  /n ( * ) 3  I |  dt  (5-18) 

0 

<  nM  x 


By  iteration,  using  (5-17)  there  results 


uUx)\  \  < 


nil  ll{  x3*' 

TT+TT 


(5-19) 


Hence,  the  series  T  ||u^+,(x)  -  u^(x)||  converges  uniformly  on  Ir,  which  in 

Jm  o 

turn  implies  that  u^(x)  converges  uniformly  on  lft  to  a  continuous  vector  func- 
tion  u(x)  which  satisfies  (5-14).  The  uniqueness  u(x)  can  be  established  as 
follows*  Suppose  v(x)  is  another  solution  of  (5-14)  in  D  on  Ir, •  Then,  it 
follows  that  (making  use  of  the  Lipschitz  condition) 


x 

I  I u^+ 1 ( x )  -  v  ( x  )  |  |  -  v  ( t )  |  |  dt 

0 


Again,  using  the  fact  that  ||u°(x)  -  v ( x )  |  |  <  nm  x,  by  iteration 


IM*Mx) 


*x)\\ 


nk  k{  x*" 

-  Tj  tut 


(5-20) 


(5-21) 


which  on  letting  j  —  oo  implies  that  ||u(x)  -  v(x)||  <  0.  Hence,  u(x )  =  v ( x ) 
on  Irt  and  the  solution  is  unique*  This  proves  the  following  theorem* 


Theorea  4:  If  the  hypotheses  of  Theorem  1  are  satisfied  and  in  addition 
Ki[x,  t,u},  •*.,  un]/duj  are  continuous  on  D,  then  there  exists  a  continuous 
solution  of  (5-14)  on  Irt  which  can  be  found  by  successive  approximations  a<id 
the  solution  is  unique* 


As  before,  the  solution  can  be  extended  to  Ir  or  to  the  boundary  of  D  by 
the  standard  argument*  Application  of  Theorem  4  to  (5-7)  yields  immediately 
the  following  important  theorem* 
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Theorem  5:  If  the  hypotheses  of  Theorem  2  or  Theorem  3  are  satisfied*  then 
a  physically  realizable  shaping  filter  exists  on  [0 *T]  and*  if  the  sign  in 
(5-10)  is  chosen  and  the  matrix  specified*  the  shaping  filter  is  unique. 

This  concludes  the  discussion  of  existence  and  uniqueness  of  physically 
realizable  shaping  filter. 


CHAPTER  6 

COMPUTATIONAL  ASPECTS  AND  APPLICATIONS 


6.1  COMPUTATIONAL  ASPECTS 

Since  the  engineer  is  usually  interested  in  synthesizing  shaping  filters 
for  use  in  applications,  he  is  faced  with  the  problem  of  actually  finding  the 
weighting  functions  and/or  differential  equations  characterizing  them.  Except 
for  certain  special  cases  such  as  stationary  processes  over  the  interval  (-oo^co) 
and  those  discussed  in  Chapter  4,  the  determination  of  the  /3^  ( t )  (or  the  coef¬ 
ficients  of  the  corresponding  differential  equation)  analytically  appears  to  be 
a  very  difficult,  if  not  impossible,  task.  Thus,  one  is  naturally  led  to  the 
consideration  of  computational  methods.  A  few  remarks  on  this  aspect  of  the 
problem  are  given  below.  One  of  the  practical  justifications  of  the  work  in 
Chapter  5  which  is  apparent  here  is  that  it  establishes  the  existence  of  a 
solution  and  its  uniqueness  properties  at  the  outset  of  the  problem,  thereby 
guaranteeing  that  one  is  not  trying  to  compute  something  that  does  not  exist. * 

In  view  of  Theorems  4  and  5,  one  of  the  immediate  methods  which  come  to 
mind  for  computing  the  /^(t)  from  (5-7)  is  that  of  successive  approximations. 
This,  of  course,  can  be  done  on  either  a  digital  or  an  analog  computer. 

Standard  references  on  numerical  methods  such  as  Hildebrand  [32]  discuss  the 
problem  from  the  standpoint  of  digital  computation  and  it  will  not  be  discussed 
further  here.  In  a  recent  article,  Tomovic  and  Parezanovic  [33]  have  inves¬ 
tigated  the  use  of  repetitive  analog  computers  for  solving  integral  equations 
by  successive  approximations.  The  interested  reader  is  referred  to  this  article 
and  those  referenced  therein. 

Since  the  integral  Equations  (5-7)  are  of  the  Volterra  type,  they  can 
also  be  integrated  directly  either  digitally  or  on  an  analog  computer  in  much 
the  same  manner  as  differential  equations  for  one  point  boundary  value  problems 
are  integrated.  This  is  perhaps  a  better  over-all  computational  procedure  than 
successive  approximations.  Finally,  Equations  (5-5)  can  also  be  solved  on  an 
analog  computer  by  implicit  methods. 

While  a  great  deal  of  effort  could  be  spent  on  developing  optimal  compu¬ 
tational  algorithms  for  integral  equations  of  the  type  given  in  (5-5)  and  (5-7)* 
it  appears  that  the  standard  techniques  mentioned  above  are  adequate  considering 
the  need. 

Of  course,  tbe  work  in  Chapter  5  ia  justifiable  in  its  own  right  because  of,  among  other  things, 
the  insight  it  gives  into  the  structure  of  certsin  classes  of  stochastic  processes. 


6.2  APPLICATIONS 


The  two  main  areas  of  application  of  shaping  filters  are  to  the  analysis 
(usually  on  analog  computers)  of  the  effects  of  noise  on  linear  systems  and  to 
the  design  of  linear  1  east-squares ,  smoothing  and  predicting  filters.  While 
these  applications  are  well  known,  a  short  presentation  of  them  will  be  given 
in  the  interest  of  completeness. 


The  first  application  is  to  the  problem  of  finding  the  variance  of  the 
output  of  a  linear  system  when  the  input  is  a  stochastic  process  whose  covariance 
function  is  known*  This  problem  reduces  to  the  computation  of  an  integral  of  the 
form 


az(t)  =  /  dr]  J  dr2  W(  )W(  t,Tz)r(Ty ,  r2)  (6-1) 

o  o 

where  is  the  weighting  function  of  the  system,  ^(Tt»T2^  i*  the  covariance 

function  of  the  input  process,  and  crz(t)  is  the  variance  of  the  output  process 
as  a  function  of  time.  When  a  shaping  filter  exists  for  the  input  process,  then 

P  7a 

nr,,r2)  =  J  d$t  J  dez  ws{Tttei)wa{Tttei)Hei  -  ez)  + 


n  n 


+  2  2  r{j  9{(r,)  q  (r2) 

i* i  j* i  4  i 


(6-2) 


Substitution  of  (6-2)  into  (6-1),  interchanging  the  order  of  the  integrations, 
and  integrating  out  the  5  function  yields 

t  ~  t  “  2 

cr2(t)  =  J  dd  J  drt  lf(  t0rl )  Ws  (r,  %d) 
o  L* 

dr,  f(  t  ,t  y)  qi  (r,  )  X 
t 

x  j  dr z  ir(t,r2)  (6-3) 

o 

The  term  in  square  brackets  in  (6-3)  represents  the  weighting  function,  W( t ,8), 
of  the  cascade  of  the  system  and  the  shaping  filter.  The  computation  of  the 
first  integral  in  (6-3)  is  easily  carried  out  on  an  analog  computer  by  the 
method  of  adjoint  systems  described  in  Laning  and  Battin  [ 2 0 J  when  the  system 
is  characterized  by  a  finite-order  linear  differential  equation,  the  covariance 
function  is  separable,  and  the  Wronskian  of  the  q^t)  exists  and  doesn't  vanish 
on  the  interval  of  interest.  The  other  terms  in  (6-3)  can  obviously  be  computed 


£  i 

{•i 


ij 


i 
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separately  with  part  of  the  same  analog  setup.  The  restriction  to  separable  co- 
variance  functions  and  the  nonvanishing  of  the  Wronskian  of  the  q^(t)  allows  the 
shaping  filter  to  be  simulated  as  shown  in  Figure  39  where  the  a^(t)  are  given 
by  (2-31)  and  the  F^(t)  are  given  by  [making  use  of  (2-26)] 


n 


FIGURE  3 


Note  that  no  differentiability  is  required  of  the  for  this  form  of 

simulation  which  in  turn  requires  only  continuity  of  first  derivatives  of 
p^(t)  to  guarantee  continuity  of  the  F^(t)  and*  hence*  continuity  of  the 

Fi(t). 

The  other  area  of  application  of  shaping  filters  is  to  the  design  of 
linear*  1  east  -  squares  *  smoothing  and  predicting  filters.  The  usual  formula¬ 
tion  of  this  problem  leads  to  the  Wiener-IIopf  integral  equation 

1 1 

r  f  d6  ir(  t}.6)  ru{0.  t,)  ;  t,  >  l,  >  0 
0 


^ 01 ^  *  1  *  *  J  ^ 


(6-5) 
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where  i»  the  covariance  function  of  the  desired  signal  at  the 

present  time  t,  and  the  input  at  time  t 2,  rjjl0,t2)  is  the  covariance  function 
of  the  input  at  time  8  and  time  t2,  and  I Ht%%8)  is  the  weighting  function  of 
the  desired  1  east  -  squares  filter*  When  the  input  is  a  white  noise  proceasft 
then  Fjj{8,  t z)  -  B(8  -  t2)  and  (6-5)  can  be  solved  immediately,  yielding 


(  rw(*,,*2)  ;  t,  >  t2  >  0 
\  0  '•*'<** 


(6-6) 


The  basic  approach  used  above  can  still  be  used  even  when  the  input  is 
not  a  white  noise  process  providing  there  exists  a  physically  realizable 
linear  system  (called  an  inverse  shaping  filter)  whose  response  to  the  input 
will  be  a  white  noise  process*  In  this  case,  following  the  Bode-Shannon  idea, 
the  input  is  first  operated  on  by  the  inverse  shaping  filter  yielding  a  white 
noise  process*  Treating  the  output  of  the  inverse  shaping  filter  as  a  new 
input  process  and  applying  the  result  of  the  previous  paragraph  one  obtains 


!»*!>  '  *1  >  *2  l  0 

0 


'•  *1  <  *2 


where 


(6-7) 


rM*(*l'*«)  =  /  dT  "»*  (t2'T)  PDI{t  t'T)  (6_8) 

0 

Here  WjMtj*7")  denotes  the  weighting  function  of  the  inverse  shaping  filter.* 
Hence,  the  weighting  function  of  the  least-squares  filter  is  given  by 


hm t , , 1 2 )  =  /  dT  r1(*I.r)[irj,(T,t2)]fl  (6-9) 

‘2 

For  cases  where  there  exists  a  shaping  filter  which  is  characterized 
by  a  finite-order  linear  differential  equation,  the  differentia]  equation  of 
the  inverse  shaping  filter  is  immediately  found  by  interchanging  the  role  of 
input  and  output*  Also,  in  this  case,  the  integral  in  (6-8)  is  easily  eval¬ 
uated  on  an  analog  computer*  When  is  separable,  then  W(  t  } ,  1 2  )  is 

separable  and,  assuming  the  requisite  differentiability,  the  1  east- squares 


*If  one  it  operating  over  an  infinite  interval,  then  the  inverse  shaping  filter  swat  be  atable  in 
the  usual  sense*  For  a  finite  interval,  stability  in  the  usual  sense  loses  its  weaning  and 
inpor tance. 
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filter  is  characterized  by  a  finite-order  linear  differential  equation  which 
is  easily  simulated  (or  built)  from  analog  components. 

From  the  above  it  is  clear  that  if  the  differential  equation  of  the 
shaping  filter  is  known,  then  the  solution  of  the  1  east- squares  filtering 
problem  is  greatly  simplified.  For  further  discussion  see  Darlington  (9]. 

Kalman  and  Bucy  [34]  have  given  an  alternate  solution  to  the  least- 
squares  filtering  problem  assuming  that  the  shaping  filter  for  the  signal 
is  known  and  the  noise  is  white  noise*  *  As  before,  the  solution  makes  use 
of  an  explicit  knowledge  of  the  differential  equation  of  a  shaping  filter. 

It  should  be  noted  that,  if  random  initial  conditions  are  required 
on  the  shaping  filter,  then,  the  approach  discussed  above  must  be  modified. 
Kalman  and  Bucy  claim  that  their  results  hold  for  this  case  without  modifi¬ 
cation. 

*There  are  processes  for  which  s  shaping  filter  does  not  exist  as,  for  exanple,  stationary 
processes  with  nonabsolutely  continuous  spectral  distribution  functions.  Hence,  Kelsan 
has  not  solved  all  the  problems  as  he  sometimes  claims. 
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APPENDIX 


Suppose  the  94(f)  are  linearly  independent  on  I  in  the  sense  that  for 

n 

all  finite  n  there  exists  no  set  of  constants  not  all  zero  such  that  X 

4  *  I 

ai  =  0  almost  everywhere  on  I  and  suppose  also  that  Jjq*(t)  dt  <  ®  for 

all  i.  Let  ft  be  the  closed  linear  manifold  generated  by  all  of  the  94(f); 

ft 

i«e«,  the  set  of  all  functions  of  the  form  X  o^  9 ^ ( t )  which  are  square  inte- 

4  *  I 

grable  over  I  or  limits  in  the  mean  of  such  sums  where  the  are  arbitrary  constants 
and  *  is  an  arbitrary  integer,  1  <  ft  <  ®,  Let  Hi j  be  the  closed  linear  manifold 
generated  by  all  of  the  94(f)  except  9  ^  ( t )  •  Then  Itl^C  lfl  and  by  a  well  known 
result  [ 2  4 »  p  2  3]  there  exists  a  function  / j ( f left,  Jjfj ( f )  dt  >  0,  such  that 
/j(t)  lHtjl  i.e.,  such  that  Jjfj(t)  X  9  ( t )  dt  =  0  for  all  9ft)  ettl  ^ .  Since 

for  all  i  ^  j,  this  shows  that  there  exists  a  function  fj(  f)  such  that 
9  i  ( t )  dt  =  0  for  i  Clearly  /j/j(f)  dt  >  0  and  /^  ( t )  Kl^  implies 

94(f)  dt |  >  0,  and  thus,  by  suitably  norming,  this  establishes  the 
existence  of  a  function  fj(t)  such  that  Jj  94(f)  /  j  ( t )  dt  =  Since  this 

can  be  done  for  every  9^(t),  this  establishes  the  existence  of  a  set  of  functions 
/4(f)  which  together  with  the  94 (t)  form  a  set  of  biorthonormal  functions  on  J# 
Note  that  since  fi(t)ef[  for  all  i,  fj  /J(  t )  dt  <  ®  for  all  i.  Finally,  the 
/4(f)  are  obviously  linearly  independent  on  I  in  the  same  sense  as  the  94(4). 


